Reproduce “Stress-Testing Capability Elicitation With Password-Locked Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ryan Greenblatt, Fabien Roger, Dmitrii Krasheninnikov, David Krueger
Researchers at Redwood Research and the University of Cambridge developed a "password-locked model" framework to study how hidden beneficial capabilities in large language models can be reliably uncovered. Their findings demonstrate that fine-tuning with a small number of high-quality demonstrations or through reinforcement learning can effectively elicit these concealed abilities, often approaching the model's full performance.
Key insights
- Fine-tuning is an efficient method for eliciting hidden capabilities, often requiring only a small number of high-quality demonstrations.
- The quality of available demonstrations or starting points is crucial for the success of both supervised fine-tuning and reinforcement learning elicitation methods.
- Eliciting capabilities in one task or sub-domain often generalizes to unlocking related capabilities in other domains.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.