B12 SeriesB · Industries

[Education] Cognitive Scaffolding or Mental Crutch? Memory Follow-Up, Courseware Production, and Instructor Review Gates in Adult Learning Services

In adult learning services, AI fits best as a back-office teaching assistant rather than a stand-in instructor: recording study preferences and sending review reminders with learner consent, helping prepare visual courseware, and routing every memory update, diagram, and subscription notice through human review gates.

Published
Reading time
7 min read
References
4
Blackboards and rows of seats in an old tiered lecture hall
Photo: cogdogblog / Wikimedia CommonsCC0source

Why an Unchecked Answer Box Undermines Adult Learning Operations

Consider a common scenario in an adult professional training program. Seeking to scale learner support and speed up course preparation, the provider launches an open-ended chat assistant for enrolled adults and uses generative tools to assemble illustrated slide decks and push study updates automatically. Problems soon surface on both sides of the classroom. Adults facing practice exercises ask the chat box for finished answers rather than working through intermediate steps, while the system logs conversation details and sends follow-up prompts without giving learners a clear way to inspect or delete what was remembered. Meanwhile, automatically generated slide decks reach learners with plausible-looking diagrams that misrepresent domain concepts because no instructor checked the visuals before export.

This failure mode stems from placing generative models at the front of the classroom as an autonomous teacher rather than behind the scenes as a governed teaching assistant. In adult learning services, AI fits best in the back office: recording study progress and practice preferences under explicit learner consent, scheduling opt-in review reminders, and assisting instructors in preparing visual courseware and managing material updates—with mandatory human review gates before any memory is retained or any instructional material is published. On rlab.works, all skill packs mapped to education represent transferable uses that require domain adaptation.

From the Contested Tutoring Benchmark to Generative Governance

To understand why structured back-office workflows matter more than free-form conversational imitation, it helps to revisit the empirical literature on tutoring and computer-assisted instruction. In The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring[1], Bloom (1984) examined doctoral dissertations completed by two University of Chicago education students[1], Anania (1982, 1983) and Burke (1984), comparing student learning under three conditions of instruction[1]:

  • Conventional: Students learn in a class with about 30 students per teacher, with tests given periodically for marking[1].
  • Mastery Learning: Students learn in a class with about 30 students per teacher under the same instruction (usually with the same teacher), taking formative tests for feedback followed by corrective procedures and parallel formative tests to determine mastery[1].
  • Tutoring: Students learn with a good tutor for each student (or for two or three students simultaneously)[1], followed periodically by formative tests, feedback-corrective procedures, and parallel formative tests as in mastery learning; the need for corrective work under tutoring is very small[1].

As its title makes explicit, Bloom’s paper treats one-to-one tutoring as the benchmark that group instruction should try to match. Later synthesis suggests that benchmark is smaller than often assumed. VanLehn (2011) reviewed experiments comparing human tutoring, computer tutoring, and “No tutoring” (instruction teaching the same content without tutoring), dividing computer tutoring systems by the granularity of user interface interaction into answer-based, step-based, and substep-based systems[4]. Most intelligent tutoring systems use step-based or substep-based granularities, whereas most other tutoring systems (CAI, CBT, or CAL) use answer-based interfaces[4]. It had been widely believed that as tutoring granularity decreases, effectiveness increases—specifically, that compared to No tutoring, the effect sizes of answer-based systems, intelligent tutoring systems, and adult human tutors are d = 0.3, 1.0, and 2.0 respectively[4]. VanLehn’s review did not confirm these beliefs: it found that the effect size of human tutoring was much lower, at d = 0.79, and the effect size of intelligent tutoring systems was 0.76, nearly as effective as human tutoring[4]. Our own inference, not a conclusion of the review: step-level feedback and verification may matter more than imitating the outward form of a human tutor.

Comparison: relative to no tutoring, intelligent tutoring systems and adult human tutors were commonly believed to reach d = 1.0 and 2.0; VanLehn (2011) found 0.76 and 0.79.
Large language models make explicit verification gates even more critical. Examining educational applications from student and teacher perspectives, Kasneci et al. (2023) noted that while large language models can create educational content, improve engagement and interaction, and personalize learning, they require teachers and learners to develop competencies and literacies to understand both the technology and its limitations, including the unexpected brittleness of such systems[2]. Integration requires a clear strategy and a pedagogical approach focused on critical thinking and fact-checking strategies, alongside safeguards for output bias, continuous human oversight, and potential misuse[2]. Similarly, the first global guidance on generative AI in education and research from UNESCO (2023) calls for a human-centred approach, recommends government regulation including data privacy protection, and asks institutions to validate systems for ethical and pedagogical appropriateness while building teachers' and researchers' capacity to use them properly[3].

Engineering the Back-Office Teaching Assistant: Consent, Courseware Review, and Subscription Gates

Translating these findings into software architecture means replacing direct model-to-student generation with bounded workflows that pause for human decisions. In the Education view on rlab.works, no skill pack carries a source-supported industry designation; every pack here is a transferable use that must be adapted to educational data contracts, privacy rules, and instructor review standards.

For learner follow-up and subscriptions, dedicated workflows enforce consent and review boundaries. Within Agent OS, the Remember and follow up (S06) skill pack takes user consent, preferences, and conversation records as input. It collects opt-in preferences and prepares deduplicated, redacted memory candidates before pausing at a human review gate where the learner confirms which memories to accept—retaining the ability to edit or delete them—before continuing conversations or scheduling requested check-ins and reminders that can be turned off. This flow uses the Personal memory retrieval and lifecycle component (working code from static inspection) to extract memory candidates, apply validity windows and retention caps, and support manual editing and deletion. In adult learning, original companion themes and fixed materials must be replaced; the system does not judge learning ability or psychological state, and support drafts are never sent automatically. Alongside reminders, the Open subscriptions and updates (S37) skill pack in Commerce governs material delivery. Starting from a plan, content preferences, and subscription consent, it uses Hosted subscription billing rehearsal (prototype maturity) to rehearse lifecycle events in an isolated test environment, pauses at a human review gate to confirm the plan and payment, checks entitlements and quotas, and then sends confirmed, unsubscribable updates. Course entitlements and messaging consent must be maintained separately, and rehearsal events cannot activate live entitlements.

For courseware preparation, workflows across AI Gateway and Intelligence give instructors veto power over raw visuals and assembled slides. The Connect images and clips (S16) skill pack takes an asset brief, reference images, and usage permission as input. After checking generation conditions, it pauses at a human review gate where an instructor approves the brief; once candidate visuals are retrieved, another human review gate requires the instructor to accept them or return a revision list. Rejected outputs return for revision and do not advance. Approved assets feed into the Prepare visual material (S30) skill pack, starting from reviewed text, reference assets, and usage permission. It normalizes assets, generates visual candidates with text and provenance records, and pauses at a human review gate where the instructor checks whether diagrams accurately explain the concept. Only after confirmation does it export presentation material—using Searchable presentation packaging (working code from static inspection) to bundle rendered slides with a searchable text layer. Instructors must verify conceptual accuracy, replace original character assets and templates, and ensure generated visuals serve only as illustrative diagrams rather than experimental results or historical physical evidence.

Layered diagram: Remember and follow up (S06) has the user confirm; Open subscriptions and updates (S37) confirms plan and payment; Connect images and clips (S16) approves the brief and reviews outputs; Prepare visual material (S30) checks visual content.
Every skill pack on rlab.works is a proposed composition demonstrated with synthetic data and is not yet connected to live services; component maturity ratings reflect static source inspection rather than classroom deployment.

Pre-Pilot Checks for Adult Education Leaders

Before funding an AI teaching-assistant or courseware pilot, an academic director or technology lead can apply the following checks:

  • Can learners inspect, edit, and erase stored memories? Verify that the workflow captures preferences only on an opt-in basis, pauses for learner confirmation before persisting memory candidates, and prohibits automated evaluation of learning ability or psychological state.
  • Do instructors hold a blocking gate over generated diagrams and slides? Confirm that visual briefs and slide decks cannot bypass instructor fact-checking, and that synthetic images are never presented as experimental results or historical physical evidence.
  • Are billing rehearsals and messaging consent decoupled from course access? Ensure that subscription tests run in an isolated sandbox that cannot grant live entitlements, and that learners can mute reminders or unsubscribe from updates without losing course access.

References

  1. [1]

    Bloom, B. S. (1984). The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring. Educational Researcher, 13(6), 4–16. doi.org/10.3102/0013189X013006004

  2. [2]

    Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., ... Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. doi.org/10.1016/j.lindif.2023.102274

  3. [3]

    UNESCO. (2023). Guidance for generative AI in education and research. United Nations Educational, Scientific and Cultural Organization. unesdoc.unesco.org ↗

  4. [4]

    VanLehn, K. (2011). The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems. Educational Psychologist, 46(4), 197–221. doi.org/10.1080/00461520.2011.611369

Share this note

Generating poster…

  • X
  • WhatsApp

AI implementation diagnosis

About 3 minutes · 9 questions

Used only for this diagnosis · Replies are written by AI