Reproduce “Mass Producing Training Data with Surprising Side Effects” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yaowen Ye, Jacob Steinhardt, Sarah Schwettmann, Cassidy Laidlaw
The authors introduce DSSE-1, a collection of synthetic post-training datasets designed to elicit behaviors that auditors may not predict. Their two-stage pipeline searches and filters candidates using held-out language-model judges. It contains 6,575 datasets across eight behaviors. In a study, researchers predicted an induced behavior in 63% of trials, but the collection is not a validated deployment screen.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.