Reproduce “Eliciting Language Model Behaviors with Investigator Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiang Lisa Li, Neil Chowdhury, Daniel D. Johnson, Tatsunori Hashimoto, Percy Liang, Sarah Schwettmann, Jacob Steinhardt
Researchers from Stanford, MIT, UC Berkeley, and Transluce developed a framework called Investigator Agents that trains models to automatically generate diverse, human-interpretable prompts for eliciting specific behaviors from large language models. The framework achieved near-perfect attack success rates in automated jailbreaking and effectively uncovered diverse strategies for eliciting hallucinations and aberrant behaviors.
Key insights
- Amortizing the cost of prompt search into the training of 'investigator models' enables scalable and flexible behavior elicitation in large language models.
- Combining Direct Preference Optimization (DPO) with a Frank-Wolfe inspired iterative algorithm successfully addresses mode collapse, leading to the discovery of diverse and effective elicitation strategies.
- The framework can uncover a wide range of LLM behaviors, from specific string matches to complex, rubric-defined traits, using human-interpretable prompts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.