Reproduce “Towards a Mechanistic Understanding of Propositional Logical Reasoning in Large Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Danchun Chen, Qiyao Yan, Chenpeng Wang, Liangming Pan

The paper traces how language models compute propositional reasoning by intervening on activations across layers and token positions in PropLogic-MI, a benchmark of one- and two-hop Boolean rules. It measures answer-probability changes when blocks or heads are mean-ablated. Facts, expressions, and queries influence computation in an early-to-late sequence. This finding concerns successful short symbolic tasks, not failures or longer reasoning.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.