Reproduce “Adaptively Private Next-Token Prediction of Large Language Models” on OpenResearch

About this paper

James Flemings, Meisam Razaviyayn, Murali Annavaram

Researchers from USC developed AdaPMixED, a framework for adaptively private next-token prediction in large language models (LLMs), which uses a noisy screening mechanism and data-dependent privacy analysis. This approach achieved a 1.4 PPL improvement while incurring 16 times less privacy loss compared to its predecessor and scaled to nearly 100,000 queries with moderate privacy guarantees.

Key insights

• The data-dependent privacy analysis is the most impactful component, reducing privacy loss by 4.5 times compared to data-independent analysis by accurately reflecting actual privacy leakage.
• Adaptive noisy screening conserves privacy budget by allowing a public model to handle queries that would otherwise incur disproportionately high privacy costs from the private ensemble.
• Differentially private prediction can be scaled to nearly 100,000 queries with competitive utility and moderate privacy loss, addressing a critical limitation of previous DP methods.

Summary by alphaXiv.