Reproduce “MoEcho: Exploiting Side-Channel Attacks to Compromise User Privacy in Mixture-of-Experts LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ruyi Ding, Tianhong Xu, Xinyi Shen, Aidong Adam Ding, Yunsi Fei
MoEcho identifies and exploits architectural side-channels in Mixture-of-Experts (MoE) large language models (LLMs) and vision-language models (VLMs) to compromise user privacy. The framework demonstrates that input-dependent expert activation patterns can be leveraged to infer sensitive input attributes with 99.8% accuracy and reconstruct model outputs with up to 92.8% accuracy.
Key insights
- MoE architectures' dynamic routing creates distinct, input-dependent temporal (Expert Load) and spatial (Expert Sequence) execution footprints that are highly correlated with user input and model output.
- These execution footprints can be reliably captured using common architectural side-channels on both CPUs (e.g., L1/L2 cache occupancy) and GPUs (e.g., performance counters, TLB access times).
- Translating these captured footprints enables concrete privacy breaches, allowing inference of sensitive user data, reconstruction of model responses, and reconstruction of visual input details.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.