Reproduce “Information Leakage in Embedding Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Congzheng Song, Ananth Raghunathan
Research demonstrates that widely used embedding models are vulnerable to information leakage, allowing reconstruction of original input words with F1 scores up to 74.33% and inference of sensitive attributes like authorship with high accuracy. The study also evaluates adversarial training as a partial defense mechanism against these privacy attacks.
Key insights
- Embeddings, despite their abstract nature, are not inherently private and can systematically leak substantial sensitive information, including original input words and hidden attributes.
- Dual-encoder models, commonly trained with contrastive learning, inadvertently capture and leak sensitive attributes like authorship due to their inherent learning objectives.
- Adversarial training can effectively mitigate sensitive attribute leakage with minimal impact on embedding utility, though its effectiveness against inversion attacks may involve more significant utility trade-offs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.