Cheaper Calls, Costlier Outcomes: Cost per Delivered Success in Agent Workflows

Cheaper Steps, Flat Budgets: The Accounting Mismatch in Agent Workflows

Consider an engineering team that optimizes an agent workflow by replacing action selection with a smaller, faster, lower-priced model. On gateway dashboards that track unit metrics, cost per call and step latency drop sharply. Yet when finance and operations reconcile monthly spend against completed work orders, expenditure per finished job may barely move—and reviewers may spend more time untangling stalled runs or reversing false completions. Unless cost is tracked per completed outcome, exception handling and human oversight can quietly consume unit-level savings.

This paradox stems from an accounting mismatch rather than model-routing pricing alone. Agent cost is usually reported per call or per attempt, whereas agents deliver outcomes through sequences of decisions, recovery, and completion checks. Cheaper, faster decision models cut per-decision cost sharply, yet auxiliary reasoning, longer interactions, failed recovery, and premature completion absorb those savings. Once failed attempts enter the numerator and only accepted outcomes count in the denominator, cost per delivered success can stay flat or rise. Evaluation must therefore be cost-aware and repeated across runs, while verification and progress preservation work better as system components than as prompt requests.

Why Faster, Cheaper Decisions Do Not Automatically Yield Cheaper Outcomes

Three studies on agent workflows and evaluation explain how unit savings evaporate across multi-step execution.

Ling, Yu, Chen, and Fang (2026) studied five agent families across two settings—text household tasks (134 tasks) and mobile app use (116 task types)—running each configuration three times across 4,500 task executions to compare original language-model action selection with a fast, low-cost structured decision model^[1]. At the decision level, mean decision cost dropped by 66.1% to 78.4% and decision time fell by 73.9% to 81.5% across all five agents; although tokens per decision rose by 53.5% to 134.9%, lower token pricing still yielded large per-step savings^[1].

At the workflow level, however, auxiliary reasoning and additional interactions absorbed those decision savings. Success stayed close to baseline on household tasks (trailing by 1.49 and 1.74 percentage points) but dropped by 13.22 and 18.10 percentage points for two mobile agents and by 2.30 percentage points for a third^[1]. Measuring cost per delivered success—total cost across all attempts including failures divided by successes—showed reductions in four agent families, but an observed mean increase of 7.7% in one mobile agent, rising from 68.70 to 73.97 thousandths of a US dollar per success even though its cost per attempt fell by 17.1% (because the 95% confidence interval spans zero, this represents an observed mean rather than a proven effect)^[1]. For another mobile agent, time per delivered success rose from 239.4 seconds to 377.8 seconds^[1].

Two failure modes drove this gap: failed recovery and premature completion. In one configuration, 2.0% of executions consumed 53.3% of inference expenditure through unsuccessful recovery^[1]. In persistent-loss cases, 40 of 57 executions with the fast decision model showed failed recovery versus 3 of 57 baseline executions^[1]. With cross-trial reflection, the fast-decision variant had higher first-trial success (383/402, or 95.27%, versus 376/402, or 93.53%) but recovered only 11 of 19 initial failures versus 25 of 26 for the baseline, ending with lower final success^[1]. Premature completion also appeared: in one example, an agent clicked a switch labeled “Off” and claimed success while Wi-Fi stayed off^[1]. The authors conclude that workflows must preserve progress, support recovery, and verify completion, assessing value through task delivery, recovery behavior, and resources per delivered outcome^[1].

These results align with Kapoor, Stroebl, Siegel, Nadgir, and Narayanan (2024), who show that a narrow focus on accuracy without attention to other metrics has produced needlessly complex and costly agents and mistaken conclusions about where accuracy gains come from^[2]. Conflating model-developer and downstream-developer benchmarking needs, inadequate holdout sets that let fragile agents take shortcuts or overfit, and a lack of standardization have caused pervasive reproducibility problems; the authors propose jointly optimizing cost and accuracy and show an optimization that greatly reduces cost while maintaining accuracy^[2].

Wiedmann, Girrbach, Schmid, and Akata (2026) reinforce this system view on a benchmark of four scientific tasks where a coding agent must find and operate a published specialist model, studying five configuration parts—task information, reasoning, self-verification, time budget, and backbone model—across more than 18,000 trajectories^[3]. On that benchmark, approximately 54% of outcome variance came from repeating the same configuration rather than changing it, showing substantial run-to-run variability^[3]. Task information provided to the agent has the largest effect, exceeding time budget and model size while reducing cost and improving calibration; more time helps only when the agent has sufficient information or a capable enough model^[3]. Crucially, prompting an agent to verify its answer has little effect on verification behavior, whereas providing a dedicated verification tool changes that behavior substantially; the authors conclude that agents should be evaluated as configurable systems and that some behaviors are better implemented in the system than requested by prompting^[3].

Engineering for Cost per Accepted Outcome Across AI Gateway and Agent OS

In our view, teams should tag every model and tool call with a task and run identifier, roll failed and retried attempts into the task ledger, and define the denominator as outcomes accepted at a human delivery gate rather than runs where the agent claims completion.

In the AI Gateway module on rlab.works, the skill pack Check usage and costs structures cost control into four steps: check budget, submit and collect batch results, compute request costs, and review discrepancies at a human gate. It is transferable to public services and finance; in finance, its adaptation boundary covers operating costs for research tools—requiring one billing owner and unified units—and never financial trade settlement. Two components at working maturity support this ledger:

• Usage reservations, budgets, versioned pricing and settlement: Reserves and settles usage, enforces budgets, and reconciles priced or unpriced ledger evidence; in our view, hard budgets are how teams cap what failed recovery loops can spend.
• Gateway usage, financial, reliability and fleet analytics: Tracks requests, latency, errors, cache savings, usage, and cost. In our view, analytics should roll failed and retried runs into the parent task to report failure-spend share and time per accepted outcome.

In the Agent OS module, the skill pack Resume interrupted work preserves progress through four steps—recover history, confirm the resume point at a human gate, resume the run, and check the result at a human gate—backed by the component (working maturity) Recoverable agent runtime and context management, which provides admission control, context budgets, persisted events, checkpoints and replay, and reconnectable runs. Transferable to consumer apps, its adaptation boundary requires that imported history never trigger automatic new execution or billing; resuming requires native checkpoints or explicit human confirmation of a new task.

Finally, the skill pack Plan and deliver work—transferable to manufacturing and used elsewhere as a general pattern—places human gates at scope approval and delivery review. In our view, completion verification should run as a dedicated tool step before the delivery gate, and only outcomes accepted at that gate should count in the denominator.

All referenced skill packs are proposed compositions demonstrated on synthetic data, not live customer deployments, and require domain-specific wiring within their stated adaptation boundaries.

Three Pre-Deployment Checks for Cost-Aware Agent Workflows

Before adopting a cheaper decision model or scaling an agent workflow, engineering and finance leaders can apply three checks:

• Include failed runs in the numerator and human acceptance in the denominator: Roll all retries and failed recovery branches into the task ledger, divide total spend by outcomes accepted at the human delivery gate, and track failure-spend share and time per accepted outcome.
• Implement checkpoints and verification as system components: Replace prompt-level verification requests with dedicated verification tools and runtime checkpoints so recovery preserves progress and imported history never triggers automatic re-billing.
• Provide structured task information and repeat evaluation runs: Because one study attributed about 54% of outcome variance to repeating the same configuration, repeat trials before comparing configurations and prioritize task information before buying larger models or longer time budgets.