Replacing Names Is Not Hiding People: Pseudonymisation and Inferential Leakage Before Text Reaches an External Model

Names Become Placeholders, Yet Documents and Replies Still Leak

Consider staff sending a document to an external model to draft a reply, or asking an assistant with persistent memory to book travel with a third party. A common safeguard is reversible pseudonymisation: personal values are swapped for placeholders before the request and restored locally in the answer. An assistant with memory may also be told in its system prompt not to disclose private details.

Both rest on assumptions: detection misses nothing in the document, placeholders survive the answer, and a reply that never names a sensitive attribute does not reveal it. Two recent studies test them: one measures what still leaks after placeholders and what survives the round trip^[1]; the other measures what an assistant’s replies imply even when they state nothing sensitive^[2].

Per-Document Arithmetic and Volunteered Accommodations: Two Failure Channels

The first channel is literal leakage per document and round-trip failure. As Beskorovainyi (2026) notes, if detection is complete the external service sees placeholders instead of literal values (though quasi-identifiers may remain), but a single missed name or identifier defeats the safeguard^[1]. The benchmark tested five open tools, a regular-expression baseline and a thin engine built on the two strongest tools, in Russian and UK English, against thresholds fixed in advance: at least 99% recall on structured identifiers, 95% on names, addresses and organisations, leaks in no more than 2% of documents, and 99% of placeholders restored after an answer^[1]. The final held-out set of 1,000 synthetic documents per language was registered before the run, and two small local models wrote a summary and a reply from the cleaned text to test the round trip^[1].

On external datasets no existing tool met all thresholds; identifier recall was 96–98% against the 99% bar^[1]. On the held-out documents, the best existing tools left a whole value visible in 61% (Russian) and 58% (UK English) of documents, and still in 41.2% and 38.3% of documents without added noise^[1]. Part of this is arithmetic: a held-out document holds about 30 (Russian) or 26 (UK English) values, so the per-document leak rate grows with length even at fixed recall^[1]. The engine, which adds checksum rules, whole-value masking and masking of every later mention of a value found once, leaked in 0.3% and 0.2% of documents—but on documents from the same generator it had been developed on over six earlier cycles^[1]. In UK English it restored at least 99% of placeholders after 760 answers; in Russian only 94.6% of name placeholders came back correctly, mainly because of wrong surname inflection, and about a quarter of Russian answers contained at least one failed name^[1]. Models also invented placeholders that were never issued (85 in one model’s Russian answers on the final set); these cannot be restored and stay visible^[1]. The limits matter: all data were synthetic and real documents were not tested; the engine’s author ran the benchmark (a declared competing interest, mitigated by pre-registration and released data); the leak metric counts literal values only, not quasi-identifiers or inference^[1].

Text without literal values still carries cues. On real online forum profiles, Staab, Vero, Balunović and Vechev (2024) showed that earlier-generation models inferred attributes such as location, income and sex with up to 85% top-1 and 95% top-3 accuracy, at about 1/100 of the cost and 1/240 of the time humans needed; text anonymisation and model alignment did not stop this^[3]. EU data-protection law draws a matching line: under Regulation (EU) 2016/679, pseudonymised data that could be attributed to a person with additional information remain information on an identifiable person (Recital 26; definition in Article 4(5))^[4]. In that framework, placeholders lower exposure but do not take data out of scope^[4].

The second channel is inference from outbound replies. Huang, Fu, Meng and Chaudhuri (2026) studied a benign setting with no attacker: an assistant with persistent memory replies on a user’s behalf to a third party about lodging, transport, meals or scheduling^[2]. Across 950 instances (10 synthetic personas, 10 sensitive attributes covering health, religion and finances, 10 routine tasks), only the 98 attribute–task pairs that can be completed without the sensitive memory were scored^[2]. A separate model played an “honest-but-curious” recipient that guessed the attribute from the reply alone, adjusted against a control without the sensitive memory; five frontier models were tested under a general and a privacy-aware system prompt^[2]. In one example, a hotel booking for a user with diabetes never names the condition but asks for an in-room refrigerator for medical supplies and a low-sugar breakfast, and the coordinator can infer it^[2].

Inferential leakage exceeded direct leakage for all five models under both prompts, by 16.8 percentage points on average for one-guess inference^[2]. With the privacy-aware prompt, three models reached near-zero direct leakage (<1%), yet inferential leakage stayed at 16.2–25.2% on top-3 guesses (13.0–21.0% on one guess)^[2]. Removing the attribute’s name from stored memory barely helped (20.6% vs 21.0%)^[2]. The authors note that many cues come from accommodations the agent volunteers although the task does not need them. They suggest stating needs with minimal detail (asking about a refrigerator without mentioning medication), getting the user’s approval before sending a reply that could still reveal an attribute, and filtering memories by task necessity or rewriting replies in a separate step^[2]. The evidence comes from synthetic personas, a simulated recipient, ten routine tasks and English^[2].

Engineering Mapping on rlab.works: Access Rules, Memory Minimisation, and Outbound Review

In our engineering view, placeholders should be treated as lowering exposure rather than anonymising: access rules still decide which documents may reach an external service, residual leaks and restoration are measured on your own documents, invented or altered placeholders are flagged before restoring, memories are filtered by task necessity, and replies to third parties that touch sensitive categories go to the user for confirmation.

Across AI Gateway and Agent OS on rlab.works, three proposed skill packs and three components map these controls:

• Access rules and traces: In AI Gateway, Set access rules (S14) runs Define access scope → Validate and review changes (human gate) → Apply access rules → Review request evidence (human gate), with Prompt registry, cache and semantic policies (prompt templates, intent classification, guardrail evidence) and Redacted conversation trace capture (schema-validated, redacted traces). The pack is transferable to the public sector (connect organisation identity and actual document permissions; prompt wording alone cannot enforce access). In our view, this is where the document classes allowed to reach an external service are decided.
• Task-minimised memory: In Agent OS, Remember and follow up (S06) runs Collect opt-in preferences → Prepare memory candidates → Let the user confirm (human gate) → Continue or route for support → Follow up as requested, with Personal memory retrieval and lifecycle (memory candidates, retention windows, manual edit and delete). The pack is source-proven for consumer apps (crisis guidance is not professional support; users decide whether to send support drafts) and transferable to education (do not infer learning ability or mental health). In our view, its memory and confirmation steps are where unneeded sensitive memories can be kept out of a draft.
• Approved handoffs: In Agent OS, Intake and handoff (S09) runs Confirm account and role → Create the service record → Approve data access (human gate) → Hand over and record. The pack is transferable to healthcare (connect hospital identity and case systems for administrative service only, without symptom-based triage). In our view, Approve data access (human gate) can be the confirmation point before sensitive details are handed on.

All three packs are proposed, unintegrated compositions on synthetic data; the three components have “working” static-inspection maturity, and none implements reversible pseudonymisation with local restoration, per-document leak measurement, or inferential-leakage testing—requirements we draw from the papers. Nothing here is legal, security, or medical advice.

Three Pre-Deployment Checks

• Score whole-document leakage and restoration: Fix thresholds before testing on your document lengths, scoring zero-leak documents, inflected-name restoration, and invented placeholders.
• Filter memories by task necessity: Withhold unrelated sensitive memories before drafting and state practical needs with minimal detail.
• Gate outbound replies on human confirmation: Flag volunteered accommodations touching health, religion, or finances for user approval before sending.