Reproduce “Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tommaso Green, Martin Gubri, Haritz Puerto, Sangdoo Yun, Seong Joon Oh
Research from Parameter Lab, NAVER AI Lab, and academic institutions demonstrates that the internal reasoning traces of Large Reasoning Models (LRMs) frequently leak sensitive user data, contrary to assumptions of privacy. These reasoning traces are identified as a new privacy attack surface, with simple prompt injection attacks extracting additional private information in nearly a quarter of cases.
Key insights
- Increasing Test-Time Compute (TTC) for reasoning improves utility but paradoxically expands the privacy attack surface by making reasoning traces more prone to leakage.
- LRMs largely disregard explicit instructions to anonymize sensitive data within their reasoning, effectively treating these traces as uncontrolled "hidden scratchpads."
- A fundamental tension exists where enhancing model reasoning capabilities for utility can inadvertently heighten the risk of exposing sensitive user data in internal thought processes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.