AI Automation ·
Does passing Google's synthetic-web tests justify delegating Korean order workflows to AI?
On November 19, 2026, Google's Chrome security team described Mirror Dimension, a synthetic web for evaluating browser agents without exposing the public internet to their actions. Its design combines isolation with resetting the environment between runs.[1] A WIRED report led us to the original announcement.[3] Our question was whether passing such an evaluation justifies delegating order changes in a Korean business. The investigation separates safety during testing from evidence about safety in deployment.
Repeatable tests answer only part of the delegation question
Consider a Korean shop operator evaluating an assistant for address changes and cancellation requests. This is a proposed application, not a business we interviewed or an incident we observed. The operator must reduce processing time without letting the assistant change more than the customer authorized. Testing on real orders can itself cause harm; comparing different initial order states can also make competing designs impossible to judge.
Our research question is whether repeatable, isolated tests containing untrusted material that encourages out-of-scope actions, followed by separate Korean integration checks, reveal more incorrect state changes than happy-path checks alone. No detection improvement has been measured here. We narrowed the question to a testable comparison that does not automatically grant production write access after a simulation passes.
The sources describe different boundaries of safety
Google's announcement supports a claim about isolation and state reset, including evaluation of an agent's resistance to instructions embedded in untrusted site content.[1] That makes failures easier to reproduce under controlled conditions. It does not establish deployment safety on a representative sample of Korean orders, mobile screens, or payment methods. Control of an environment and representativeness of an environment are different properties.
For a Korean reference point, Toss Payments documents virtual test approvals without actual withdrawal, alongside differences from live operation, including live-only settlement records and payment-method-specific support.[2] A successful simulated payment therefore cannot establish that settlement has been verified. These are payment integration documents, not browser-agent security benchmarks. We compared the limits of two kinds of testing rather than treating the products as equivalent competitors.
Neither source independently validates the other. Google describes its own engineering approach; Toss Payments documents its own integration conditions. This is document research. We did not conduct penetration tests, make payments, or interview users. The reduction in real operational errors remains unknown.
| Decision-relevant item | Google's overseas example | Documented Korean condition |
|---|---|---|
| Operator and source date | Chrome security team announcement, November 19, 2026[1] | Toss Payments developer documentation, checked November 20[2] |
| Controlled test boundary | Synthetic sites and reset initial state between runs[1] | Virtual approvals without withdrawal and specified integration conditions[2] |
| What a pass cannot establish | Representative safety for Korean order workflows remains unverified | Live-only settlement and method-specific differences are not eliminated[2] |
The comparison changes the implementation proposal. Test how the agent treats instructions in an authorized, business-owned replica, and check payment integration against the provider's supported test conditions separately. Neither layer replaces the other. We do not propose attempting live automated payments to fill a gap in test coverage.
Alternative A: separate order-state evaluation from integration checks
Create authorized replica screens and fictional orders. Compare ordinary content with untrusted content that encourages changes outside the task. An evaluator should compare the customer's requested modification with the actual stored state, rather than relying on the assistant's claim that it succeeded. Check payment-related behavior only through supported test facilities in a separate stage.
This could make failures reproducible and help distinguish interface errors from instruction-following failures. Its costs include state-reset tooling, replica maintenance, and human assessment. A simplified replica could create false confidence. The operator must record differences from the real workflow and leave unsupported functions explicitly unverified. Starting with dummy inputs that require no copied customer data or authentication material is an editorial proposal.
A pilot is reasonable only where the business can define permitted state transitions and maintain the replica. Owning a realistic-looking screen does not ensure that the underlying cancellation or recovery rules are realistic. The experiment must examine state, including what happens after an interrupted attempt.
Alternative B: let AI prepare the work and use a fixed change procedure
Keep the assistant's role to reading and drafting a proposed action. A person authorizes a change through a fixed form or an existing business API. This reduces the range of interface choices delegated to the agent. It may suit rapidly changing screens, teams unable to maintain replicas, or workflows with costly cancellation and settlement recovery.
The costs are review queues and staff time, and people can approve incorrect proposals. Conversely, Alternative A does not provide meaningful oversight if the reviewer sees only a final confirmation button. Neither approach has a demonstrated lower price in the reviewed sources. Compare engineering and maintenance time, recovery work, and customer support costs. Stable recurring tasks with resettable state and reliable change histories could justify a limited pilot of A.
Measure unauthorized state changes before expanding access
The business owner should define allowed changes first. Developers then prepare fictional orders and reset procedures; a separate reviewer authors expected final states. Compare A with happy-path-only testing on matched order types. Evaluate B on the same types for elapsed processing time and staff correction effort. These are proposed procedures, not experiments we performed.
Proposed measures include out-of-scope field changes, missing or duplicated changes, recovery after failure, unsupported coverage, rejection of valid requests, and total human review time. A headline success rate should disclose which workflows enter its denominator. We propose stopping the affected write path whenever an out-of-scope change is reproduced, then reviewing its cause and expected outcome. This is a proposed conservative rule, not an observed incident count or a vendor acceptance standard. An unsupported function remains unverified.
The answer to the opening question is conditional. Synthetic environments address external side effects and repeatability during evaluation; they are not sufficient evidence for approving Korean order delegation. Business-specific state transitions and gaps in Korean integration support need separate verification. Independent comparisons on representative Korean-language and mobile workflows could strengthen or overturn this judgment. The remaining question is which real failure modes the simulation still cannot express.