Reproduce “Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Usman Anwar, Tim Bakker, Dana Kianfar, Cristina Pinneri, Christos Louizos

The authors introduce a reward that encourages a model’s chain of thought to predict its output, aiming to improve monitoring without behavior labels. Their theory makes positive conditional mutual information necessary but not sufficient: information must concern the monitored attribute and be extractable. In Qwen2.5-7B-Instruct MBPP experiments, monitor accuracy rose from 28.46% to 88.89%, but monitoring was not guaranteed across tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.