Reproduce “Hidden Commitment: When Language Models Silently Pick a Side and How Steering Can Surface It” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Samuel Dawit Assefa, Jae Won Cho
The authors study how models choose between conflicting retrieved documents and disclose conflicts. In Qwen 0.8B, they derive a source-choice direction from residual states and steer it: positive steering raises Document 2 choices from 51.5% to 74.7%, while negative steering lowers them to 32.2%; random-direction controls remain near baseline. This tracks document choice, not correctness, and disclosure steering recovers only some failures.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.