Reproduce “LLMs Can Understand Encrypted Prompt: Towards Privacy-Computing Friendly Transformers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xuanqi Liu, Zhuotao Liu
Researchers at Tsinghua University developed a privacy-preserving inference framework for Large Language Models by replacing cryptographically challenging non-linear functions with efficient approximations. This approach achieved a 5x acceleration in computation time and an 80% reduction in communication overhead compared to the state-of-the-art Iron framework, while maintaining model accuracy.
Key insights
- Non-linear functions like GELU, Softmax, and Layer Normalization are the dominant performance bottlenecks in private inference for transformers, collectively accounting for over 70% of the total time and communication costs.
- Systematic, incremental operator substitution combined with fine-tuning and bound controlling can effectively replace cryptographically unfriendly functions while preserving model accuracy.
- Designing neural networks to be 'privacy-computing friendly' at the architectural level offers a robust path towards practical private AI, beyond relying solely on cryptographic optimizations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.