Reproduce “REER-PT: Reverse-Engineered Reasoning for Perplexity-Guided Pre-training Data Augmentation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haoran Que, Jiajun Shi, Ting Huang, Renming Pang, Jiaheng Liu, Ge Zhang, Wenhao Huang, Shen Yan and 2 more
REER-PT introduces a perplexity-guided, offline framework for augmenting raw pre-training data with explicit, 'book-note-style' reasoning annotations. This method improved continuation predictability and enhanced downstream performance of language models on knowledge and various reasoning benchmarks, with gains up to 2.07 percentage points on tasks like BBH and GPQA-Diamond.
Key insights
- Perplexity serves as an effective optimization signal for identifying specific content gaps and refining generated reasoning annotations to improve a language model's ability to predict subsequent text.
- Targeted, concise reasoning annotations, which explicitly articulate the conceptual or discourse connections, can enhance the predictability of unchanged source content.
- Offline, scalable data augmentation with explicit reasoning can be achieved by prioritizing annotations that demonstrably reduce the perplexity of observed continuations, filtering out less useful reasoning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.