Reproduce “Tracing the Thought of a Grandmaster-level Chess-Playing Transformer” on OpenResearch

About this paper

Rui Lin, Zhenyu Jin, Guancheng Zhou, Xuyang Ge, Wentao Shu, Jiaxing Wu, Junxuan Wang, Zhengfu He and 2 more

A unified sparse decomposition framework for grandmaster-level chess Transformers reveals dynamic reasoning pathways by interpreting both MLP and attention modules. The research demonstrates that internal computations for different candidate moves largely rely on disjoint sets of features, and decision-critical information progressively converges onto move-relevant squares in deeper layers, while also identifying specific tactical logic and potential model ambiguities.

Key insights

• LC0's grandmaster-level reasoning incorporates rich, human-interpretable tactical logic, such as cross-layer information transfer for piece control and assessment of opponent's defensive coverage.
• The model exhibits a highly distributed reasoning paradigm where different candidate moves predominantly utilize distinct sets of significant features (low path overlap).
• Decision-critical information becomes progressively concentrated on fewer squares and specifically converges onto the source and target squares of the final move in deeper Transformer layers.

Summary by alphaXiv.