Reproduce “Training Data Extraction From Pre-trained Language Models: A Survey” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shotaro Ishihara
A comprehensive survey by Shotaro Ishihara at Nikkei Inc. systematically reviews training data extraction from pre-trained language models, consolidating over 100 papers to categorize memorization definitions, attack methodologies, and defense strategies. The review highlights how factors like model size and data duplication empirically correlate with increased memorization.
Key insights
- Memorization definitions are diverse, ranging from exact "eidetic" to "approximate" matches and formally privacy-guaranteed "differential privacy" concepts.
- Training data extraction attacks often follow a two-step process: generating numerous candidate texts followed by membership inference using metrics like perplexity.
- Memorization is empirically influenced by factors like larger model sizes, higher data duplication rates, and longer prompts, often showing log-linear relationships.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.