Reproduce “Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Adam Karvonen, Benjamin Wright, Can Rager, Rico Angell, Jannik Brinkmann, Logan Smith, Claudio Mayrink Verdun, David Bau and 1 more
Karvonen et al. introduced novel supervised evaluation metrics and a training technique for Sparse Autoencoders (SAEs) by using language models trained on board game transcripts like Chess and Othello. This approach provides a quantifiable, objective framework to assess SAE feature quality against known ground-truth board states, demonstrating that a new 'p-annealing' method improves feature learning and reduces shrinkage compared to standard SAEs.
Key insights
- Language models trained on board games offer a unique, controlled environment where interpretable features can be formally defined, enabling objective and supervised evaluation of SAEs.
- The new supervised metrics provide a more granular and discerning assessment of SAE quality and progress compared to traditional unsupervised metrics like L0 and loss recovered.
- The p-annealing training method can enhance SAE performance, achieving results comparable to more complex architectures like Gated SAEs while being computationally more efficient.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.