Reproduce “A Sense of Proprioception in Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jonas Geiping
The paper trains models to report checkable facts about internal computations, using labels read from the same forward pass and conversational on-policy training. In a mixture-of-experts routing task, the model reports which expert ranked highest for a token and layer. It reaches 62.8% accuracy, versus 19.0% for a lookup baseline, but fails to identify routing changes caused by tiny floating-point differences.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.