E32 SeriesE · Organisation & people

Skill Compression and the Risk of 'Experience Hollowing': How AI Flattens the Novice Learning Curve and Shifts Senior Experts Toward Architecture and Final Review

Experimental evidence across customer support, professional writing, and software development shows that generative AI boosts novices more than senior staff; if junior roles routinely bypass foundational training, organizations risk a future shortage of qualified reviewers unless career ladders and workflows are redesigned.

Published
Reading time
7 min read
References
4
An instructor and trainees beside machines in the metalwork shop of a vocational training centre
Photo: Lauri Veerde / Wikimedia CommonsCC BY-SA 4.0source

Faster Novices Today, Fewer Reviewers Tomorrow?

Consider a hypothetical service center or advisory team introducing conversational and drafting assistants. Newly hired analysts quickly produce fluent responses and initial reports, narrowing the visible gap with seasoned staff. Yet when an unusual edge case or compliance boundary arises, junior staff struggle to spot subtle errors in machine-generated drafts and rely on senior leads. If entry-level roles routinely bypass verifying facts, organizing records, and decomposing problems, who will possess the judgment required to serve as final reviewers when today’s novices are promoted?

Experimental evidence from customer support, professional writing, and programming shows that generative AI boosts novices more than senior workers. Our inference, not a proven finding, is that if junior roles skip foundational training for long enough, organizations risk “experience hollowing” and a shortage of qualified reviewers, so career ladders and workflows need redesign.

Empirical Evidence on Skill Compression and "Shadow Learning"

The “skill compression” effect—where generative AI narrows performance gaps across experience tiers—appears across field and controlled experiments.

Studying the staggered introduction of a generative AI conversational assistant across 5,172 customer-support agents, Brynjolfsson, Li, and Raymond (2025) found that AI access increased productivity, measured by issues resolved per hour, by 15% on average, with substantial heterogeneity across workers[1]. Less experienced and lower-skilled workers improved both speed and quality, while the most experienced and highest-skilled workers saw small speed gains and small quality declines[1]. AI assistance also facilitated worker learning, improved English fluency—particularly among international agents—and improved the work experience: customers were more polite and less likely to ask for a manager[1]. Gains from AI adoption were largest for moderately rare problems, where human agents had less baseline experience but the system still had adequate training data[1].

Controlled experiments in writing and coding show similar patterns. Noy and Zhang (2023) conducted a preregistered online experiment assigning occupation-specific, incentivized midlevel professional writing tasks to 453 college-educated professionals, randomly exposing half to an assistive chatbot[2]. Average time taken decreased by 40%, output quality rose by 18%, inequality between workers decreased, and concern and excitement about AI temporarily rose[2]. Exposed workers were 2 times as likely to report using it in their real job 2 weeks after the experiment, and 1.6 times as likely 2 months after[2]. In software development, Peng et al. (2023) ran a controlled experiment where recruited developers implemented an HTTP server in JavaScript as quickly as possible[3]. The treatment group with an AI pair programmer completed the task 55.8% faster than the control group, and heterogeneous effects showed promise for helping people transition into software development careers[3].

Three cards: AI assistance raised support productivity by 15% on average, most for novices; professional writing took 40% less time with 18% higher quality and less inequality; AI pair programming made a coding task 55.8% faster.
Does accelerating novices hollow out deep expertise over time? Currently, "experience hollowing" is an inference supported only indirectly by Beane's (2019) study of robotic surgery[4]. Through a two-year, five-sited comparative ethnographic study of robotic and traditional surgical practice and blinded interviews at 13 top-tier U.S. teaching hospitals, Beane (2019) examined how trainees build skill when approved learning practices fail[4]. Learning through increasing participation worked well in traditional open surgery, but robotic surgery greatly limited trainees' role and made approved methods ineffective[4]. Only a minority of trainees reached competence through "shadow learning"—norm- and policy-challenging practices enacted extensively, opportunistically, and in relative isolation[4]. Successful trainees relied on three practices[4]: "premature specialization" in robotic technique at the expense of generalist training; "abstract rehearsal" before and during rotations when concrete rehearsal was prized; and "undersupervised struggle," performing close to the edge of their capacity with little expert supervision when norms and policy dictated it[4]. These unpunished practices contributed to troubling outcomes for initiate surgeons and the profession, including hyperspecialization and a decreasing supply of experts relative to demand[4].

By analogy, if organizations deploy AI solely to speed up drafting—letting juniors skip verification while disconnecting seniors from review—they risk a similar bottleneck in future expert supply.

Engineering the Ladder: Verification and Review Gates in Workflows

To preserve skill formation, workflows should embed junior verification and senior review as explicit state-machine steps rather than relying on open-ended chat. Across the Agent OS and Intelligence modules, this division of labor is structured through skill packs:

  • Permissioned intake (Intake and handoff S09): From Service request, identity assertions and assignment rules, the workflow assists operators to Confirm account and role and Create the service record. It pauses at Approve data access (human review by the assigned reviewer or requesting user; rejected output returns for revision and does not advance), then packages the minimum approved data for Hand over and record.
  • Source alignment (Turn long audio into notes S19): Taking Authorized recording, media metadata and note requirements, the system runs Prepare recording sources and Transcribe and align to output positioned transcripts with replica offsets. A human gate—Check text and attribution—requires verifying text and speaker attribution before assisted Draft the notes and final human confirmation at Review and export.
  • Evidence review (Research and write a report S24): From Research question, permitted sources and report outline, the pack assists with Plan the search scope, Collect and extract material, and Organize citations and findings. At Review the evidence (human review), reviewers check sectioned material and source indices—returning rejected claims for revision—before Format and export.
  • Senior architecture (Plan and deliver work S01): Taking Goal, scope and delivery requirements, senior leads anchor human gates: after assisted Plan the steps, they Approve the scope before Run and collect, then Review and hand over against execution evidence, while Reusable skill extraction, evaluation and solution packs records reuse patterns. Auditable workflows, human confirmation and decision review and Collaborative delivery, approvals and evidence gates manage state pauses, drafts, and evidence gates.
Two columns: juniors keep non-bypassable checks (Check text and attribution S19, Review the evidence S24, Approve data access S09); seniors approve the scope and review the handover in S01, logging return reasons as coaching feedback.
Honest engineering requires clear adaptation boundaries. None of these packs has a source-supported industry tag in business and general services, healthcare, legal services, or education; they are transferable uses or general patterns only. S09 and S19 are transferable uses in healthcare: S09 helps a booking desk verify permissions and collect minimum data for administrative handoff, but must connect to hospital identity and ticketing systems, supports administrative service only, and never performs automated symptom triage. S19 turns consented booking interviews into timestamped notes for administrative staff to check names and dates, serving only as a service record—not for diagnosis or automated chart judgment—with strict retention and access limits. In legal services, S24 is a transferable use for retrieving approved case files and statutes with preserved citation positions for legal review, requiring replacement of paper ingest and news topic models, verification of statutory validity and case applicability, and no automated legal conclusions. S01 is transferable only in manufacturing and serves as a general pattern in the industries here.

All skill packs on the site are proposed compositions demonstrated with synthetic data and not yet connected to live services; component maturity reflects static source inspection rather than production deployment. Auditable workflows, human confirmation and decision review is a prototype: it persists workflow steps and pauses for human confirmation, while tool steps remain placeholders and records and agent suggestions come from inputs or defaults rather than autonomous execution.

Pre-Pilot Checks for Redesigning the Organizational Ladder

Before funding an AI pilot, a CEO, CDO, or department head can apply concrete checks:

  • Do junior workflows keep non-bypassable verification gates? Ensure juniors must align sources, verify citations, and assemble minimum data packages before generating drafts.
  • Have senior roles shifted toward scope architecture and final review? Verify that seniors hold blocking authority at scope and delivery gates, logging rejection rationales as training feedback.
  • Are escalation rules set for rare and boundary cases? Separate routine inquiries from high-stakes boundaries—such as clinical triage or legal validity—requiring immediate specialist handoff rather than undersupervised struggle.

References

  1. [1]

    Brynjolfsson, E., Li, D., & Raymond, L. R. (2025). Generative AI at Work. The Quarterly Journal of Economics, 140(2), 889–942 (NBER Working Paper No. 31161, 2023). doi.org/10.1093/qje/qjae044

  2. [2]

    Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187–192. doi.org/10.1126/science.adh2586

  3. [3]

    Peng, S., Kalliamvakou, E., Cihon, P., & Demirer, M. (2023). The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. arXiv preprint arXiv:2302.06590. arxiv.org ↗

  4. [4]

    Beane, M. (2019). Shadow Learning: Building Robotic Surgical Skill When Approved Means Fail. Administrative Science Quarterly, 64(1), 87–123. doi.org/10.1177/0001839217751692

Share this note

Generating poster…

  • X
  • WhatsApp

AI implementation diagnosis

About 3 minutes · 9 questions

Used only for this diagnosis · Replies are written by AI