Policy in the Prompt Is Not Enough: Execution-Grounded Policy Guards for Permitted-Tool Violations

Why Putting Deployment Policy in the System Prompt Fails to Stop Permitted-Tool Violations

When enterprise teams move tool-calling agents toward production, they commonly define an allowlist of callable tools and paste operational compliance rules—spending caps, folder boundaries, cross-department limits, or prerequisite sign-offs—into the task agent’s system prompt, assuming a restricted tool catalog plus prompt self-regulation will enforce policy.

In practice, the hardest failures rarely involve calling a banned tool. Instead, an agent invokes an explicitly permitted tool—such as an approved database update, file-editing utility, or messaging service—while breaking an argument threshold, targeting an out-of-scope resource, or acting before a precondition is met. Because the tool sits on the allowlist, tool-identity filtering sees no issue, and the task agent still lets violating calls slip through.

This exposes a structural gap: an allowed tool is not a compliant tool call, and putting deployment policy in the task agent’s prompt leaves substantial enforcement error. Closing that gap requires training, scoring, and enforcing a separate pre-execution policy guard on environment-verified calls rather than relying on prompt self-regulation alone.

From Executable Environments to Coevolution: Policy Benchmarks and Separate Guards for Permitted Tools

The need for pre-execution checks follows from how language-model agents interact with external systems. As Yao et al. (2023) established in the ReAct paradigm, tool-calling agents interleave reasoning traces with concrete action proposals; because those proposed actions trigger real side effects once dispatched, compliance verification must inspect the proposed call before execution^[3].

To measure and strengthen enforcement when violations occur on otherwise-allowed tools, Shang et al. (2026) introduce AgentPolicyForge, which uses executable environments to construct policy benchmarks and iteratively train adaptive policy guards^[1]. Rather than relying on ungrounded text labels, the framework executes tool calls inside real environments and records outcomes to help label compliance^[1]. Using this pipeline, the authors build PolicyGym across ten policy families, covering 698 scenarios for training-data generation and 254 separate held-out scenarios for human-reviewed testing (187 in-domain and 67 out-of-domain scenarios within the executable environment suite)^[1]. Every violating test case in PolicyGym invokes a permitted tool—detection requires checking call arguments, target resources, or context conditions rather than tool identity alone^[1]. These benchmark counts reflect controlled evaluation in executable environments, not production incident prevalence^[1].

AgentPolicyForge trains a separate guard through coevolution: an attacker generates requests using retrieved strategies, the guard is fine-tuned on balanced, environment-verified examples, and archive updates feed subsequent rounds^[1]. The comparisons in Shang et al. (2026) show why prompt self-regulation and static baselines fall short:

• Self-regulation baselines: Without policy in the prompt (policy-blind self-regulation), attack success rate (ASR) exceeds 98% on threat episodes^[1]. When deployment policy is placed in the task agent’s prompt (policy-aware self-regulation), in-domain and out-of-domain ASR still reach 14.77% and 13.16%, with 8.79% in-domain and 9.21% out-of-domain benign rejection (over-blocking)^[1].
• Prior compact guardrail baseline: In the binary comparison table reported by Shang et al. (2026), a prior 4B-parameter compliance guardrail baseline from Wen et al. (2026) records 32.52% in-domain ASR and 2.80% benign rejection^{[1, 2]}.
• Ten-round coevolutionary trained guard: After ten rounds of coevolutionary training, a separate 14B trained guard filtering fixed task-agent proposals reaches 8.22% in-domain and 7.89% out-of-domain ASR, with 7.48% in-domain and 5.92% out-of-domain benign rejection^[1]. Compared with policy-aware task-agent self-regulation, this final guard configuration achieves a 40–44% relative reduction in ASR and fewer benign rejections^[1].

An archive ablation further shows that across ten matched rounds, an evolving strategy archive cuts in-domain ASR from 10.84% (with a frozen initial archive) to 8.22%—a 24.1% relative reduction—at +0.75 percentage points more benign over-blocking^[1]. Interpreting the comparison between self-regulation and the separate guard also requires stating a caveat noted by the authors: the guard filters fixed task-agent proposals, whereas a policy-aware task agent can revise its actions in-loop, so the two configurations are not identical control loops^[1].

Engineering Mapping on rlab.works: Separating Tool Inventory, Prompt Policy, and Pre-Execution Gateways

In our engineering view, these findings point to six rules: keep deployment policy as structured data; evaluate pre-execution guards on environment-verified violating calls that still use allowed tools; prefer a separate pre-execution guard over self-regulation alone; refresh attack archives across iterations; measure benign over-blocking alongside ASR; and never equate “tool allowed” with “call compliant.”

Across Agent OS and AI Gateway on rlab.works, three proposed skill packs and three working components illustrate how these layers fit together—and where their adaptation boundaries lie:

• From prompt policy registries to separate gateway checks: In AI Gateway, Team access rules (S14) (also titled Set access rules) runs define access scope → validate and review changes (human gate) → apply access rules → review request evidence (human gate) to bind policy to tools, resources, and conditions. First, Semantic policy prompt registry manages prompt templates and cache, classifies intent, and emits context and guardrail evidence (with advisory context optimization). Because prompt-held policy is necessary but insufficient on its own, the application step routes tool traffic through a separate MCP policy gateway (proxying MCP over HTTP, SSE, or stdio with endpoint, tool, and namespace policies, process restrictions, and audit metadata) and a Governed provider and tool gateway (exposing registered REST, MCP, and SDK operations with scoped identity, argument and budget checks, approvals, idempotency, and execution evidence). Marked transferable in public services (requiring organizational identity and document permissions rather than prompt wording alone), applying this pattern in enterprise, legal, or financial workflows to govern argument thresholds and resource scopes requires domain contract wiring.
• Pairing machine checks on permitted tools with human gates: In Agent OS, Approve computer actions (S03) runs check readiness (identity and action scope) → approve the action list (human gate) → act and retain evidence → check the changes (human gate). Transferable in legal services, it requires restricting read and write folders explicitly and keeping external filing outside default permissions—pairing machine checks on permitted tools’ arguments and context with a human gate before side effects.
• Separating tool onboarding from call compliance: In Agent OS, Install and manage tools (S04) runs check packages and licenses → approve account scopes (human gate) → install and register → check calls and rollback (human gate), and is transferable in enterprise settings (selecting adapters by package format and client while retaining failure and revocation paths). Curating a tool inventory and OAuth scopes governs which tools may be installed, whereas call-time argument and resource compliance still requires a pre-execution guard.

Current boundaries must be stated candidly. All three components carry a working rating from static source inspection only (live integrations were not executed, and credential-bearing configs are excluded); they provide gateway policy checks and prompt-policy management, and do not implement the AgentPolicyForge training pipeline or PolicyGym benchmark. Likewise, all three skill packs are proposed compositions demonstrated on synthetic data, requiring domain contract and identity wiring before enterprise, legal, or financial use.

Three Pre-Deployment Checks for Execution-Grounded Policy Guards

Before authorizing tool-calling agents in operational workflows, engineering and risk leaders should apply three checks:

• Enforce argument and resource rules in a separate pre-execution guard: Keep prompt templates to guide planning, but deploy a separate guard or gateway that verifies arguments, target resources, and context conditions on every call to an allowed tool.
• Score guards on environment-verified permitted-tool violations and refresh archives: Test guards where allowed tools are called with violating arguments or unmet preconditions, ground labels in executable-environment outcomes, and regularly update the adversarial strategy archive.
• Measure attack success and benign over-blocking together, backed by human gates: Track both violation leakage and false rejections on every guard update, routing high-impact or ambiguous calls to a human review gate with audit evidence.