Reproduce “Differentially Private Next-Token Prediction of Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
James Flemings, Meisam Razaviyayn, Murali Annavaram
A protocol for differentially private next-token prediction in large language models, called PMixED, is proposed by researchers at USC, tailored for black-box API access. It demonstrates improved utility over DP-SGD, reducing perplexity by up to 3 points on standard benchmarks while providing strong group-level privacy guarantees.
Key insights
- Differential Privacy can be effectively achieved during the black-box inference phase of LLMs by leveraging inherent output stochasticity and a public auxiliary model, addressing the adversary model mismatch.
- The introduction of (alpha, beta)-RD Mollifiers provides a novel theoretical framework for projecting privacy-sensitive distributions onto a set around a public distribution, crucial for the privacy mechanism.
- An ensemble-based private prediction approach offers a better privacy-utility trade-off and enhanced computational efficiency compared to private training methods like DP-SGD, especially for practical privacy budgets.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.