Reproduce “PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models” on OpenResearch

About this paper

Haoran Li, Dadi Guo, Donghao Li, Wei Fan, Qi Hu, Xin Liu, Chunkit Chan, Duanyi Yao and 2 more

PrivLM-Bench introduces a multi-level privacy evaluation benchmark for language models, empirically quantifying privacy leakage for both fine-tuning and inference data through established attack methodologies. The benchmark demonstrates that standard Differential Privacy tuning effectively protects fine-tuning data but largely fails to safeguard inference data privacy, providing a critical insight into real-world model deployment.

Key insights

• Standard DP-tuning effectively protects fine-tuning data but provides minimal to no protection for sensitive information contained within inference data, revealing a critical gap in current PPLM practices.
• Privacy vulnerabilities are highly dependent on the model architecture and size, with parameter-efficient tuning methods (e.g., prompt tuning) showing inherent robustness against certain attacks even without explicit DP.
• Empirical privacy attacks can reveal different privacy performance than theoretical DP guarantees, suggesting that worst-case adversary assumptions might sometimes overestimate actual attack capabilities.

Summary by alphaXiv.