Reproduce “PrivacyXray: Detecting Privacy Breaches in LLMs through Semantic Consistency and Probability Certainty” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jinwen He, Yiyang Lu, Zijin Lin, Kai Chen, Yue Zhao
Researchers from the Chinese Academy of Sciences developed PrivacyXray, a framework for identifying accurate private information leaked by Large Language Models during inference. By analyzing internal states for semantic consistency and probabilistic certainty, the method achieves an average accuracy of 92.69% in detecting privacy breaches across various LLM architectures.
Key insights
- Correctly generated private outputs exhibit distinct internal patterns, including narrower, higher-peaked sentence-level probability distributions and stable, high-confidence token probabilities across layers.
- Accurate private outputs demonstrate consistently high inter-layer semantic similarity in decision token embeddings, indicating stable semantic trajectories, and high intra-layer semantic coherence among top-ranked tokens.
- Leveraging these internal state observations enables the detection of semantically equivalent privacy leakage, extending beyond traditional exact string matching methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.