Reproduce “ALGEN: Few-shot Inversion Attacks on Textual Embeddings using Alignment and Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yiyi Chen, Qiongkai Xu, Johannes Bjerva

ALGEN, developed by researchers at Aalborg University and Macquarie University, introduces a few-shot inversion attack on textual embeddings, drastically reducing the data required to reconstruct original text. It demonstrates the ability to initiate partial inversions with a single data point and achieve optimal performance with 1,000 samples against various black-box and proprietary embedders, highlighting a critical privacy vulnerability in systems using textual embeddings.

Key insights

  • Textual embedding spaces from different black-box models can be effectively aligned to an attacker's space through a one-step linear optimization, even with very limited data.
  • Existing defense mechanisms like noise insertion, shuffling, or differential privacy prove largely ineffective against few-shot inversion attacks without significantly degrading the utility of the embeddings.
  • Cross-domain and cross-lingual inversion attacks can still recover meaningful and sensitive information, such as Named Entities, from embeddings, even if the reconstructed text requires translation.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.