Reproduce “Text Embedding Inversion Security for Multilingual Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yiyi Chen, Heather Lent, Johannes Bjerva

Researchers at Aalborg University demonstrated that multilingual language models are vulnerable to text embedding inversion attacks, revealing that current noise-based defenses are ineffective in multilingual contexts. They proposed a simple masking defense that significantly reduces text reconstruction fidelity while preserving embedding utility for both monolingual and multilingual models.

Key insights

  • Multilingual LLMs are highly susceptible to embedding inversion attacks, and in some cases, can be even more vulnerable than their monolingual English counterparts.
  • Existing noise-based defenses, while effective for English models, prove largely inadequate in protecting multilingual models against inversion attacks.
  • A simple masking defense, by targeting a specific embedding dimension for language identification, offers a robust and effective countermeasure for both monolingual and multilingual models without sacrificing utility.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.