Reproduce “Large Language Models Are Cross-Lingual Knowledge-Free Reasoners” on OpenResearch

About this paper

Peng Hu, Sizhe Liu, Changjiang Gao, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang

Researchers from Nanjing University and China Mobile Research decompose large language model reasoning into knowledge retrieval and knowledge-free reasoning components, finding that knowledge-free reasoning achieves near-perfect cross-lingual transfer, while knowledge retrieval significantly hinders it. Their interpretability analysis points to shared neural mechanisms for knowledge-free reasoning across different languages.

Key insights

• Abstract, knowledge-free reasoning is inherently language-agnostic, enabling robust and near-perfect cross-lingual transfer in LLMs.
• Reliance on an LLM's internal knowledge base for retrieval actively impedes its ability to transfer reasoning skills across languages, as knowledge storage appears more language-specific.
• The model's foundational linguistic proficiency in the target language is a critical prerequisite for effective cross-lingual transfer of knowledge-free reasoning capabilities.
• Knowledge-free reasoning tasks exhibit significantly higher computational similarity across languages in terms of hidden states and neuron activations, particularly in middle layers, suggesting shared neural mechanisms.

Summary by alphaXiv.