Ending "Hallucinated ROI": True Cost Accounting and Value Attribution for Enterprise AI Projects

When Theoretical Minutes Saved Become Phantom Financial Returns

Consider a familiar situation in an enterprise pilot review. A business unit reports that after introducing a generative assistant, employees draft copy and assemble spreadsheets faster in trials. Sponsors multiply those saved minutes by headcount and wage rates to project a large return. Yet when quarterly results arrive, invocation bills have climbed, conversion and retention show no verified improvement, and senior staff spend extra hours inspecting and rewriting flawed upstream drafts.

Treating local drafting speedups as realized profit creates “hallucinated ROI.” Our thesis is simple: AI project ROI must include invocation costs, retries, and human rework, and must be anchored to business outcomes verified through controlled comparison; “theoretical minutes saved” cannot be booked directly as financial return.

From Task Speedups to Aggregate Output: Task Boundaries, Heterogeneity, and Oversight

Why does faster completion of an individual step fail to scale linearly into organizational performance? Macroeconomic theory and workplace studies point to task complexity, worker heterogeneity, and oversight costs.

At the macroeconomic level, Acemoglu (2025) evaluates AI’s aggregate implications using a task-based model working through automation and task complementarities^[1]. So long as microeconomic effects stem from task-level cost savings or productivity gains, macroeconomic consequences follow a version of Hulten’s theorem: GDP and aggregate productivity gains can be estimated from the fraction of tasks impacted and average task-level cost savings^[1]. Using existing estimates of AI exposure and task-level productivity gains, these macroeconomic effects are non-trivial but modest—no more than a 0.66% increase in total factor productivity (TFP) over 10 years^[1]. Even these estimates may be exaggerated because early evidence comes from easy-to-learn tasks, whereas future effects will partly come from hard-to-learn tasks where context-dependent factors shape decisions and no objective outcome measures exist from which to learn successful performance; consequently, predicted TFP gains over the next 10 years are less than 0.53%^[1]. Theoretically, even when AI raises low-skill workers’ productivity in certain tasks without creating new tasks, inequality may rise; empirically, while AI impacts are more equally distributed across demographic groups than previous automation technologies, there is no evidence AI will reduce labor income inequality, it is predicted to widen the capital-labor income gap, and some new tasks (such as algorithms for online manipulation) may have negative social value^[1].

Inside the firm, task-level gains are equally uneven. Brynjolfsson, Li, and Raymond (2025) studied the staggered introduction of a generative AI conversational assistant across 5,172 customer-support agents^[2]. AI assistance raised worker productivity, measured by issues resolved per hour, by 15% on average, with substantial heterogeneity across workers^[2]. Less experienced and lower-skilled workers improved both speed and quality, whereas the most experienced and highest-skilled workers saw small speed gains and small quality declines^[2]. AI assistance also facilitated worker learning and improved English fluency, particularly among international agents; yielded the largest gains on moderately rare problems where human agents had less baseline experience but the system had adequate training data; and improved work experience as customers were more polite and less likely to ask for a manager^[2]. Extrapolating from a narrow pilot without controlling for worker tenure and problem distribution distorts return estimates.

The AI Index 2024 Annual Report (Maslej et al., 2024) reinforces these boundaries^[3]. AI has surpassed human performance on several benchmarks (some image classification, visual reasoning, and English understanding) but trails on complex tasks such as competition-level mathematics, visual commonsense reasoning, and planning^[3]. While several 2023 studies suggested AI lets workers finish tasks faster, improve quality, and narrow the skill gap, other studies caution that using AI without proper oversight can diminish performance, and robust, standardized evaluations for LLM responsibility are seriously lacking^[3].

Engineering Value Attribution: Cost Ledgers, Metric Alignment, and Conversion Feedback

Defensible value attribution requires controls across both costs and outcomes. On rlab.works, this discipline spans the AI Gateway, Intelligence, and Commerce modules through structured skill packs and human review gates.

In AI Gateway, [Check usage and costs (S13)](/skills/check-usage-and-costs/) checks budget before submitting batch jobs and reconciles costs against recorded usage. From an input of batch jobs, pricing rules, and available budget, it checks available budget via [Usage reservations, budgets, versioned pricing and settlement](/capabilities/usage-budget-pricing-ledger/) to reserve usage, enforce limits, and output verified funding records and reservations; submits and collects results (returning task results and provider usage receipts); and accounts for request costs (reconciling priced or unpriced evidence into a traceable ledger). At the Review discrepancies human review gate, the designated reviewer or requesting user inspects the ledger—rejected outputs return for revision and do not advance—yielding a human-confirmed bill or flagged discrepancies. For finance and for government and public services, this pack is a transferable use: in finance, it links a research department's batch jobs to a cost ledger to explain unpriced and over-budget discrepancies for research tool operating expenses with a chosen billing owner and unified units, excluding financial transaction settlement; in government and public services, it books a public inquiry assistant's batch jobs against departmental budgets for month-end review of unpriced and duplicate charges, requiring fiscal workflow mapping and claiming no regulatory compliance. It has no tag for media and content or travel and hotels, serving only as a general pattern.

In Intelligence, Review weekly operations (S26) prevents mistaking seasonal noise or shifting definitions for AI gains. Taking authorized business tables, comparison periods, and previous reports as input, it collects and aligns tables (outputting unified data with provenance, definitions, and ownership) and compares periods and segments (outputting cross-period trends and segment comparisons). It then finds exception candidates via Order-to-traffic anomaly diagnostics, drilling down to exact and dashboard-consistent definitions to flag traffic-to-order divergence, partner concentration, geographic mismatch, and low-base effects. At the Check causes and evidence human review gate, a reviewer verifies explanations—rejected outputs return for revision—before assembling the archived weekly report and follow-ups. For travel and hotels, this pack is a source-supported industry—aligning orders, room nights, visits, and channel data while retaining definitions and tracking prior issues—though warehouse permissions and definitions must be verified and holiday overlap does not prove causation. It has no tag for finance, government and public services, or media and content.

In Commerce, Review campaign results (S39) connects content output to verified actions. Using published content, permitted analytics, and campaign goals as input, it invokes Attribution and conversion feedback to link tracked campaign URLs with external analytics and proprietary registration and order data. It links sources and actions (outputting a traceable visit and action table), checks performance and health (outputting content comparisons, missing data, and runtime anomalies), and drafts next-round adjustments. At the Choose the next plan human review gate, a reviewer must approve the plan—rejected drafts return for revision—before scheduling and retaining execution history and the next baseline. For media and content, this pack is a transferable use, aligning article referral links, reading, and subscription actions to guide topic and channel selection; teams must distinguish reading, subscription, and paid events, and attribution correlation does not prove causal reach. It has no tag for finance, government and public services, or travel and hotels.

Every skill pack on rlab.works is a proposed composition demonstrated with synthetic data and not yet connected to live services. The components above are working code assessed via static source inspection rather than production deployment, and a source-supported industry tag means only that source data has an explicit industry association—making no claims about customers, deployments, or ROI.

Pre-Pilot Checks for Defensible Value Attribution

Before funding or scaling an AI pilot, executives and department heads can apply these checks:

• Does cost accounting include retries, discrepancies, and human rework? Require batch jobs to run against a reservation and reconciliation ledger, and count senior staff time spent reviewing discrepancies and revising rejected drafts.
• Are outcome evaluations anchored to controlled periods and cohorts? Lock metric definitions and baselines up front across worker skill levels and task types to filter out low-base effects, partner concentration, and holiday noise.
• Do attribution workflows gate next-round plans on verified business actions? Reject ROI cases built on theoretical minutes saved; require traceable links from outputs to ticket resolution quality, subscriptions, or orders, with a blocking human review gate before plans advance.