Reproduce “From Trait Vectors to Circuits: Tracing Refusal and Sycophancy Through Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Oscar Miró López-Feliu, Maya Ozbayoglu
The authors distinguish circuits that reconstruct a trait coordinate from circuits that transmit an imposed steering signal, then test whether restoring the coordinate recovers behavior. In Qwen2.5-7B-Instruct, a 10,699-edge two-stage refusal circuit retains 90.63% refusals on 64 held-out prompts, versus 51.56% for an equal-budget direct circuit. This supports the comparison under the tested conditions, not a unique natural mechanism.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.