PrivacySkills Preprint Examines How Guidance Influences LLM Agent Privacy Choices
An arXiv preprint dated September 28 2026 introduces PrivacySkills a framework that measures how system instructions and skill level labels affect whether LLM agents select…
- Without privacy guidance and with users available agents accessed confidential sources in 30 percent of valid runs on average across five open weight models.
- Combining system level instructions with skill level intrusiveness labels roughly halves confidential access rates.
- Skill level metadata labels alone yield a modest reduction to 24 percent on average while urgency framing shows no detectable effect.
The PrivacySkills Framework
The preprint presents PrivacySkills as a controlled evaluation setup built around 55 synthetic tasks spanning 11 categories of personal information. Each task is paired with 169 skills that outline different acquisition pathways delivering identical task relevant value: consulting public sources accessing confidential data or querying the user directly.
The study tests three forms of privacy guidance: system level instructions skill level metadata labels indicating intrusiveness or both. It also varies user availability and urgency framing across experiments run on five open weight models.
Key Results from the Preprint
When users are available and no privacy guidance is supplied agents choose confidential sources in 30 percent of valid runs on average. This rate rises to 45 percent when users are unavailable. Urgency framing has no detectable impact.
System level privacy instructions by themselves produce limited reduction in confidential access. Skill level intrusiveness labels alone lower the rate modestly to 24 percent on average. Combining the two approaches roughly halves the rate of confidential source selection.
This summary is based solely on the preprint abstract. It does not reflect a full paper review or independent replication.
Implications for Agent Privacy Design
The authors argue that incorporating privacy annotations directly into skill specifications alongside system level instructions deserves further study as a way to shape agent behavior. Privacy outcomes appear sensitive to how guidance is presented rather than arising automatically.
The work highlights the importance of deliberate design in LLM agents that handle personal information and motivates continued evaluation of annotation effectiveness.
Limitations Noted in the Abstract
Findings are limited to the specific 55 synthetic tasks and the five open weight models evaluated. Outcomes could differ in real world deployments with varied tasks models or implementations.
The preprint stops short of offering universal solutions and instead calls for additional research on privacy annotations paired with prompts.
