Reproduce “Correctness Conditioning as Selection in Chain-of-Thought Faithfulness Evaluation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yueer Pan, Yuexi Pan
The authors examine how filtering for original correctness changes intervention-based chain-of-thought faithfulness scores. In an arithmetic example, two rules both answer 11, but after an intermediate value changes, one recomputes to 9 while the other repeats 11. In GLM-5.3 arithmetic tests, correct-only scoring is 58.1%, versus 50.7% across all traces; this finding is limited to arithmetic tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.