Reproduce “TRAP: Understanding and Mitigating Privacy Memorization in Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Muhammed Ustaomeroglu, Ziyue Xu, Hanshen Xiao, Peter Cnudde, Guannan Qu, Holger R. Roth

The authors introduce TRAP, a fine-tuning penalty that compares a target model’s token predictions with those of a reference trained on complementary data, penalizing tokens the target predicts better. On a synthetic rare-secret mixture, TRAP with rank-64 LoRA lowered worst-case nAUC from 0.51 to 0.14, while cross-entropy rose from 0.0070 to 0.0084 nats; this empirical result is not a privacy guarantee.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.