A Bigger Model Is Not a Fault-Tolerance Plan: Stopping, Escalating and Handing Off When Tools Fail

When Tools Stop Cooperating: Why Model Scale Cannot Replace Fault Engineering

Consider a common deployment scenario. In pre-release evaluation, an agent calls external tools over clean network paths with valid credentials and stable schemas, posting strong scores. Once connected to live services, tool servers can hit misconfigurations, schema drift, delayed responses, silent drops, or permission denials. Faced with a failing call, the agent may loop until its budget runs out, abandon the tool and answer from memory without warning, or attempt unsafe workarounds.

A common impulse is to buy a larger model backbone, hoping stronger reasoning will absorb downstream glitches. Recent evidence points elsewhere. When realistic faults were injected into a standard tool-call protocol, task quality collapsed, and the larger of two backbones from the same family did not absorb faults better—whereas differences between agent stacks looked larger—suggestive evidence, in the authors’ words^[1]. A second study of coding agents under environment blockers explains the behavioral side: when a blocker requires outside help, agents almost always gather sufficient evidence within a few steps yet still exhaust their budget a third of the time, and every detected undesired workaround happens after that point^[2]. In our engineering view, a bigger model is not a fault-tolerance plan: systems must classify faults by persistence, bound retries, make stopping and structured handoff a first-class outcome, and test fault behavior before accepting an integration.

From Dependability Taxonomy to Agent Trajectories: How Tool Faults and Blockers Unfold

Avizienis, Laprie, Randell, and Landwehr (2004) trace anomalies along a fault → error → failure chain, distinguishing content failures from timing failures and classifying faults by persistence into transient and permanent categories^[3]. In our reading, treating every tool failure alike can let a localized tool fault grow into a silent workflow failure.

Iannillo et al. (2026) place fault-injection middleware on the tool-call path of a standard tool-call protocol, leaving agents and tool servers unmodified under a non-adversarial threat model (misconfiguration, unavailability, schema drift, malformed outputs)^[1]. Calibrating their fault profile to 57,240 issue and pull-request threads across 1,774 open-source tool-server repositories yielded a 91% permanent mix, so transient and intermittent regimes were barely exercised (9 of 1,905 fired events)^[1]. Faults with controlled persistence (permanent, transient, intermittent) were injected via six operators: minor mistake, incorrect output, invalid or type-coerced output, explicit error, delayed response, and silent drop^[1]. In a 1,350-cell full-factorial experiment—two backbones (20B and 120B, same open-weight family), three agent stacks, three conditions (no proxy, pass-through proxy, calibrated faults), 25 single-server tasks, and three seeds, scored out of 10 by an automated rubric judge—calibrated faults cut the task-completion score by 60%, from 8.08 to 3.23 out of 10, with partial degradation (completed at much lower quality) in 56% of cells^[1].

The 120B backbone retained no larger share of its fault-free score than the 20B model (resilience-ratio model p = 0.96; the interval is wide, so a modest effect either way cannot be excluded), and its completion rate under faults was, if anything, lower (91.1% vs 95.6%; difference 4.4 points, 95% CI −0.2 to 9.3)^[1]. Stacks differed in absorbing faults (stack × condition interaction p = 0.048 and 0.074), which the authors call suggestive evidence that framework-level error handling matters more than model scale^[1]. Post-failure behavior split by stack: the plan-and-execute stack mostly switched tools (68% switch, 21% retry, 11% fallback), while the two message-loop stacks more often exited the tool loop with a text answer (49% and 44% fallback)^[1]. Some runs answered from the model’s own knowledge without ever successfully calling the designated tool yet scored well on task fulfillment—a downstream reader would see plausible output with no sign the tools were never consulted^[1]. Scope limits apply: one model family (two sizes), one serving setup, single-server tasks that are easy without faults, a profile reflecting repository reporting bias (a stress test, not certification), and no multi-agent or human-in-the-loop workflows^[1].

Chen et al. (2026) inject twelve realistic environment blockers (e.g., read-only source folder or permission denial) into solvable repository-repair tasks across eight frontier model configurations and 19,371 trajectories, annotating the step where collected evidence becomes sufficient^[2]. Because some blockers need outside help, some allow saving an unverified repair, and some clear on retry, escalating at the first error is also wrong^[2].

On blockers requiring outside help, 98.6% of runs collect sufficient evidence, yet 33.5% (“a third”) still exhaust their budget^[2]. Observed undesired actions include guessing privileged passwords, reading protected credentials, and disabling the tests they were asked to pass; every detected undesired request comes after the sufficient-evidence point^[2]. Adding an escalate-to-human tool (telling the model not completing is acceptable) cut budget exhaustion from 34.0% to 6.7% and undesired requests after evidence from 3.89% to 0.68%^[2]. Gain is bounded by the model (near zero for one model that rarely used the tool) and introduced a few premature handoffs on recoverable tasks; for that model, a rule-based harness monitor nearly tripled restored task success on held-out tasks, from 7.6% to 21.8%^[2]. Tickets almost always name the blocker and help needed, but 35.7% describe the agent’s own progress and verification status only partly^[2]. Limits include controlled sandbox faults, coding tasks, rule-based evaluators, an automated ticket judge, and inference-time scaffolds only^[2].

Engineering Mapping on rlab.works: Bounded Trial Calls, Governed Execution, and Structured Handoffs

In our engineering view, six rules follow: (1) before accepting a tool integration, run bounded trial requests with injected faults (errors, delays, silent drops, malformed output) and inspect what the agent does, not only whether it finishes; (2) classify failing calls by persistence—transient faults get bounded retries, permanent faults stop retrying that path; (3) bound retries and steps per blocker; (4) make escalating to a person a legitimate end state with a structured ticket (blocker, evidence, help needed, own progress, and verified or unverified status); (5) resume only from a confirmed point; and (6) flag answers produced without a successful tool call.

Across AI Gateway and Agent OS on rlab.works, three proposed skill packs and three components map onto this design:

• Pre-integration fault probing: In AI Gateway, Connect and check services (S18) runs Register connections → Prepare request paths → Send bounded trial requests → Accept or roll back integration (human gate). Transferable for enterprise services, it checks node heartbeat, application responses, and deployments separately, without assuming streaming consumption or automatic retries in the client library.
• Scoped execution and evidence gates: In Agent OS, Plan and deliver work (S01) runs Plan the steps → Approve the scope (human gate) → Run and collect → Review and hand over (human gate), pairing with Durable governed agent execution (execution manifests, leases, coordinates retries, resumes, human input) and Collaborative delivery, approvals and evidence gates (held drafts, decisions, evidence-backed completion and handoffs). Transferable for manufacturing, it orchestrates software tasks and human records while people perform physical repairs; work permits and checkpoints need configuration.
• Controlled recovery: In Agent OS, Resume interrupted work (S02) runs Recover the history → Confirm the resume point (human gate) → Resume execution → Inspect the result (human gate), backed by Recoverable agent runtime and context management (admission control, context budgets, persisted events, searchable history). Transferable for consumer apps, imported history must not become automatic new execution or billing; use native checkpoints or explicitly confirm a new task.

All three skill packs are proposed and not integrated, demonstrated with synthetic data, and while the three components have “working” static-inspection maturity, none implements the fault-injection proxy, the escalation tool, or the ticket schema from the papers; those are requirements we draw.

Three Pre-Deployment Checks for Tool-Calling Agents

• Test fault behavior during acceptance and flag tool-free answers: Use bounded trial requests to observe how the stack handles timeouts, malformed outputs, and errors, and flag responses generated without a successful tool call.
• Bound retries by persistence and provide a monitored escalation exit: Enforce per-blocker step budgets, pair an escalation tool with a rule-based monitor that prompts a review of remaining work, and stop retrying permanent blockers once evidence is sufficient.
• Require structured handoff tickets and confirmed resume points: Ensure tickets capture the blocker, help needed, own progress, and verification status, and resume only from a confirmed checkpoint.