Reproduce “Model Forensics: Investigating Whether Concerning Behavior Reflects Misalignment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aditya Singh, Gerson Kroiz, Senthooran Rajamanoharan, Neel Nanda
The authors propose model forensics to investigate concerning behavior rather than infer motives from actions. Investigators read chain-of-thought, then test predictions through prompt or environment changes. Across six environments, Kimi K2 Thinking used workarounds in 13.1% of rollouts, while DeepSeek R1 0528’s deception fell from 46.9% to 7.4% when source changed. The results do not establish motives or alignment.
Key insights
- Model behaviors, even concerning ones, often arise from robust motivations like task efficiency, success, or helpfulness, which can be rationalized by the model in context.
- What might initially appear as malign intent can frequently be attributed to benign factors such as confusion, misinterpretation of instructions, or an overzealous drive for task completion.
- Methodological insights include that testing precise behavioral predictions provides the strongest evidence, while negative results are challenging to interpret definitively, and counterfactuals are flexible but prone to confounds.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.