Reproduce “Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Haonan Duan, Adam Dziedzic, Nicolas Papernot, Franziska Boenisch

Researchers at the University of Toronto and Vector Institute developed the first differentially private prompt learning algorithms for Large Language Models (LLMs), empirically demonstrating prompt data vulnerability and providing solutions for both soft and discrete prompts. Their PromptPATE method, designed for black-box LLMs, achieved 92.7% accuracy on the sst2 dataset with (ε = 0.147, δ = 10^-6)-differential privacy, closely matching non-private baselines.

Key insights

  • LLM prompts are vulnerable to privacy attacks like Membership Inference Attacks, revealing a critical, unaddressed privacy vector.
  • Differential privacy can be effectively integrated into prompt learning, offering a more parameter-efficient and practical alternative to full model fine-tuning.
  • The PATE framework can be adapted for black-box LLMs using an ensemble of discrete "teacher prompts," enabling private knowledge transfer and the creation of a differentially private "student prompt."

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.