Reproduce “Stealing User Prompts from Mixture of Experts” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Itay Yona, Ilia Shumailov, Jamie Hayes, Nicholas Carlini
Researchers at Google DeepMind developed "MoE Tiebreak Leakage," an attack that extracts user prompts from Mixture-of-Experts LLMs by exploiting cross-batch information leakage in routing mechanisms. This method successfully recovered 99.9% of secret messages tested.
Key insights
- Mixture-of-Experts (MoE) architectures, specifically Expert-Choice-Routing, violate the implicit assumption of batch independence in LLM processing.
- The deterministic tie-breaking mechanism of torch.topk in CUDA, based on token order, creates a side channel that reveals information about co-batched user prompts.
- Performance optimizations in MoE models can inadvertently introduce a new class of architectural vulnerabilities for prompt extraction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.