Reproduce “Position: Privacy Is Not Just Memorization!” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Niloofar Mireshghallah, Tianshi Li
The paper argues that Large Language Model (LLM) privacy research has an overly narrow focus on data memorization, proposing a comprehensive taxonomy of five privacy incident categories to capture a broader spectrum of risks. An empirical analysis reveals a substantial misalignment between current research priorities and the most pressing, real-world LLM privacy threats.
Key insights
- LLM privacy risks extend significantly beyond verbatim memorization, encompassing advanced forms of indirect attribute inference, data aggregation, and various forms of leakage through autonomous agents and context flow.
- Privacy challenges in LLM systems are inherently sociotechnical, requiring interdisciplinary approaches that integrate insights from design, policy, and ethics, rather than purely algorithmic solutions.
- Current AI/ML privacy research exhibits a striking misalignment, with 92% of studies focusing on memorization or direct chat leakage, leaving critical emerging threats largely underexplored.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.