Reproduce “Differentially Private Natural Language Models: Recent Advances and Future Directions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lijie Hu, Ivan Habernal, Lei Shen, Di Wang
Researchers provide the first systematic and technical review of recent advancements in applying Differential Privacy (DP) to deep learning models within Natural Language Processing (DP-NLP). This work synthesizes approximately 70 papers, categorizing DP-NLP methods and identifying unique challenges and future research directions for building privacy-preserving language systems.
Key insights
- DP-NLP requires specialized privacy notions beyond canonical DP, such as Selective Differential Privacy for sparse sensitive information and Metric Differential Privacy to account for semantic similarity in text.
- Privacy protection in NLP can be applied at different granularities—word-level, sentence-level, or user-level—each presenting distinct technical challenges.
- The existing research landscape in DP-NLP is dominated by three methodological paradigms: gradient perturbation, embedding vector perturbation, and ensemble models, each with specific strengths and trade-offs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.