"At Least Three" Has a Price: How Minimum Counts in Extraction Schemas Make Models Pad Their Lists

When the Document Has Fewer Items Than the Schema Demands

Extraction pipelines often ask a model for JSON that follows a schema, and schemas can set a minimum list length—asking, for example, for at least three requests from every email. To catch hallucinations, many teams also pair schema validation with a substring check verifying that each entry occurs in the source document.

When the document holds fewer true items than that minimum, the request cannot be met truthfully. Barali and Paudel (2026) find that, rather than stopping at the true items, models treat the minimum as a target and pad the list. When an email with one request is asked for three, the model returns the request plus an ordinary sentence plus the sign-off^[1]. Because that padding is real text copied from the document, it parses, passes the schema, and about half of it passes a “does this entry occur in the source?” check^[1].

The engineering consequence is clear: list length must be decided by the document, not the schema; a “short list” must be a valid, reviewable outcome; and evaluation must compare against answer lists, not only schema validity or source-occurrence checks.

From Format Constraints to "Padding to k": What Models Put into Forced Slots

Format constraints are not neutral: Tam et al. (2024) observed that format restrictions (JSON/XML) produced a significant decline in reasoning ability, with stricter constraints generally causing greater degradation^[3]. As constrained decoding became the dominant way to enforce structured outputs, Geng et al. (2025) introduced JSONSchemaBench—roughly ten thousand real-world JSON schemas—to evaluate constrained-decoding frameworks on efficiency, constraint coverage, and output quality^[2]. It asks whether the decoder meets the schema^[2]; Barali and Paudel (2026) ask what the model puts in the slots the schema forces^[1]. Of 9,542 real-world schemas in JSONSchemaBench, 7.7% put a minimum of at least 1 on a list of strings or objects (post hoc analysis)^[1].

In a study preregistered before the full run, Barali and Paudel (2026) tested eight open-weight models on one GPU under greedy decoding (plus six open-weight models on hosted services as an unpooled reference) on 558 real documents from nine annotated sets drawn from eight published datasets: meeting action items, email requests, receipt items, job-ad skills and knowledge, medications in case reports, product-review features, people in encyclopedia sentences, and direct speech in novels^[1]. Documents hold 0–6 checked items, and each model was asked for 1, 3, or 5 items—stated in prose, shown in a prompt schema (not enforced), or enforced by the decoder^[1].

Their findings detail how minimum counts drive padding:

• A shown minimum raises over-counting without enforcement: A minimum shown in the schema but not enforced raises the share of lists longer than the true count (over-count) from 34.5% to 55.4%—about one more list in five running past the true count—and adds 0.62 entries per output when pooled^[1]. Six of the eight GPU models lengthen their lists (the two smallest are the exceptions; one often writes no readable list)^[1]. Even on documents a model gets exactly right with no count, a shown minimum still makes 35.2% of its lists pad (post hoc)^[1].
• Unfillable slots drive the effect, and enforcement hits short and empty documents: A minimum adds nothing on documents that already hold enough items—the effect comes from slots the document cannot fill—while a shown maximum barely changes over-count (35.7% vs. 34.5%) but shortens lists and leaves out more true items^[1]. An enforced minimum leaves 90.1% of short-document outputs padded, and on the 105 documents with no true item, even a minimum of 1 enforced by the decoder raises the share of outputs with at least one substantive entry from 34.9% to 82.4%^[1].
• Grounded padding defeats source checks: Resting on the paper’s item matcher, most extra entries are grounded padding—real text from the document that is not an item of the kind asked for, such as a sign-off line counted as a request: 62.9% under a shown minimum and 79.6% under an enforced one^[1]. About half of all extra entries under a shown minimum (46.6%, post hoc) pass a source-occurrence check, so a source check does not catch it^[1].
• Common fixes help only partly, and bare error retries worsen padding: A “fewer is fine” line (“If the document contains fewer than k, list all available ones.”) removes the effect on average (lists 0.77 entries shorter than no count), but the share of true items left out rises from 17.0% to 24.3% (matcher-based), and some hosted models still lengthen lists^[1]. Stating “up to k” in prose yields the best entry F1 among conditions yet raises over-count from 34.5% to 42.7%; an “N/A” option under an enforced minimum cuts padding only from 90.1% to 51.0%; and thinking mode did not help in exploratory runs^[1]. Re-asking with the bare validation error—as schema-validating libraries commonly do—raised over-count among re-asked outputs in every run tested, jumping from 10.0% to 69.0% for one model in exploratory testing^[1].

Scope limits apply: only open-weight models were tested (closed APIs were not tested and may behave differently); each list item is one string (multi-field records were not tested); one grammar engine was used; and how often real pipelines meet short documents was not measured^[1].

Engineering Mapping on rlab.works: Let the Document Decide Length and Surface Gaps

In our engineering view, teams should let the document decide list length (placing no minimums on document-determined lists); make “short / none found” a first-class, typed outcome that flows to a human gate or a gap list; never auto-retry with the bare validation error; record count shortfall as a gap rather than filling it; and evaluate with answer lists including short and empty documents, scoring over-count and recall together rather than only schema pass rate or source occurrence.

Across Intelligence and AI Gateway on rlab.works, three proposed skill packs and three working components map to these controls:

• Surfacing gaps instead of filling slots: In Intelligence, Unify and enrich records (S28) runs ingest → normalize/resolve entities → classify and locate gaps → investigate and accept facts (human review), surfacing missing facts for review rather than padding lists. In travel it has a source-supported association (each supplier needs a field contract, retained conflicts, and provenance, with catalog wiring still to be implemented). Underneath, Grounding coverage and repair planning identifies missing dimensions and plans targeted reruns—treating missing data as a recorded gap rather than a slot to fill.
• Human review of extracted evidence: In Intelligence, Research and write a report (S24) runs plan search → collect and extract material → organize citations → human review of evidence → export; in legal workflows it is transferable (replace sources, review legal validity, and do not automate legal conclusions). Keep an evidence memo (S25) runs collect source evidence → resolve entities → compare claims and counterevidence → human review and save memo, holding a source-supported association in finance (supporting research, not trade instructions) and a transferable pattern in legal. Upstream extraction can use Disclosure collection and text extraction on text-bearing PDFs (not OCR) before human gates filter out padded lines.
• Structured evaluation with short and empty fixtures: In AI Gateway, Task-aware model routing and structured evaluation validates structured outputs against deterministic fixtures, which can include short and empty documents.

All three components carry a working rating from static source inspection only, without executed integration tests, and none implements the paper’s study or eliminates padding. All three skill packs are proposed compositions demonstrated on synthetic data, requiring domain contracts and live wiring before deployment.

Three Checks Before Deploying Extraction Schemas

• Remove minimum counts from document-determined lists: Let schemas accept empty or short lists as valid typed outcomes that route to a gap list or human gate.
• Stop auto-retrying short lists with bare validation errors: Do not feed raw minimum-count errors back to the model; record shortfalls as coverage gaps for review.
• Evaluate over-count and recall on short and empty documents: Score extraction outputs against reference answer lists rather than relying on schema validity or source-occurrence checks alone.