AI Automation ·
Can a Korean financial firm treat a missing AI policy verdict as permission to execute?
A Google News US search for AI agents surfaced Google Developers Blog’s November 20 announcement of Semantic governance policies. The Preview checks proposed tool calls before execution against user intent and organisational rules.[1] We initially asked whether connecting this service would complete a Korean financial firm’s approval evidence. The technical documentation instead raised a prior question: what happens to requests for which no verdict exists?
Establish the evaluated population before measuring verdict accuracy
Consider a Korean financial firm assessing an agent that retrieves internal material and prepares handoffs. Security staff need to prevent improper actions; business users need legitimate work to continue. A dashboard containing only allow and deny records can omit unevaluated requests from its denominator. This is an evaluation problem, not a report of a Korean breach.
Our research question is how to assess coverage, incorrect decisions and disruption without interpreting a missing verdict as permission. We compared the original announcement, configuration and scope documentation, and Financial Security Institute’s domestic evaluation direction. We deployed no cloud environment and reproduced no attack. The operating design below is our proposal, not product certification.
Observation and blocking are different states
Google’s configuration documentation, updated November 20, warns that the LLM can err and recommends dry run before enforcement. Dry run records verdicts without blocking actions.[2] Therefore, a recorded denial and a prevented execution are different observations. Our inference is that logs must be compared with outcomes.
The configuration guide also directs operators investigating absent verdicts to engine, gateway-binding and request-path conditions.[2] The overview explicitly excludes streamed model responses and warns that calls within them might not be blocked.[3] This establishes a documented boundary, not evidence of a bypass in any Korean system.
Our question changed from whether a policy sentence was good enough to which request passed which check and corresponded to which execution. Misinterpreting a rule and never entering its evaluation path require different repairs. Combining them into one rejection rate obscures whether to change policy or integration.
Korean evaluation addresses actions as well as answers
Financial Security Institute’s October 8 announcement describes examining execution authority, approval controls, results and stopping. Year-end pilot validation and expanded formal application in 2027 are announced plans.[4] They are not evidence of completed implementation across Korean institutions or approval of Google’s product.
The comparison leads us to propose separate records for a policy verdict, an authorised person’s approval and actual system execution. We did not establish that this service alone satisfies any specific statutory approval obligation.
| Decision issue | Overseas primary material | Korean primary material |
|---|---|---|
| Status | Preview announcement for semantic checks before execution[1] | Agent-security criteria announced, with pilot and expansion plans[4] |
| Scope | Supported request paths and response formats[2][3] | Execution authority, approval, results and stopping[4] |
| Unverified here | Missing-verdict rates, errors and latency in Korean workflows | Domestic product acceptance and deployment outcomes |
This compares different roles, not equivalent institutions or proof that one satisfies the other. It changes the pilot by requiring evidence connecting policy verdicts with approvals and execution.
Option A keeps existing execution boundaries and adds observation
Retain tool permissions, explicit conditions and human approval as the execution basis while comparing semantic verdicts on simulated requests. Narrow work to retrieval and drafts, keeping real external transmission and state changes behind existing controls. Context-sensitive judgment arrives later, but coverage gaps and errors can be examined before automated blocking affects work.
People must still produce justified test labels and verify existing controls. A narrow simulation may omit real linguistic variation. A long observation period alone cannot establish representativeness.
Option B adds semantic enforcement only on verified paths
A bounded workflow with genuinely contextual rules may benefit from an added semantic check. Retain tool-side permissions and conditions. We propose holding mutations or returning them to established approval when verdicts are missing, erroneous or unsupported. This is an operating choice to verify, not a claim about the product’s default outage behaviour.
Costs include mistaken rejection of legitimate work, latency, exception review and policy-change validation, as well as model charges. Adding a judge after granting consequential execution does not dissolve responsibility. Filling missing verdicts with old permission records would defeat B’s premise.
Reconcile decisions with actual execution in the pilot
Business owners should define allowed, prohibited and unresolved cases with reasons. Security staff inventory tools, paths and response formats. Platform staff connect inputs, policy versions, verdicts, approvals and outcomes using request identifiers. Reviewers should establish reference judgments without seeing the engine’s answers first.
Use a controlled environment without real customer material or external recipients. Repeat equivalent intentions in different language; count missing verdicts separately from incorrect allows and denials. Safely simulate unsupported requests and integration failure without broadening live transmission permissions or disabling protections.
Proposed measures include matched verdict/outcome coverage across all proposed calls, missing verdicts, decision errors, repeated-decision variation, added latency and exception labour. Report denominators and sample scope. We measured none of these; acceptance criteria should be set beforehand according to loss and recoverability.
Stop expansion if an executed request lacks an explainable inspection record; repair routing and tool-side controls first. Reduce enforcement scope if false rejections overwhelm review capacity. Nor should universal human review be assumed effective without examining queues and omissions.
No verdict is an unresolved state, not a verdict value
The sources support treating absence separately and establishing both inspection coverage and execution outcomes. Contextual rules may add protection, but do not eliminate missing paths or probabilistic errors. Assessment must retain the surrounding execution boundaries.
Errors across Korean-language variations, observability of every path, actual failure behaviour and total operating costs remain unverified. Independent reproduction and complete execution reconciliation could change B’s appropriate scope. Does the dashboard’s “allowed” total contain inspected requests, or also requests that received no decision?