Reproduce “Mixture of Cognitive Reasoners: Modular Reasoning with Brain-Like Specialization” on OpenResearch

About this paper

Badr AlKhamissi, C. Nicolò De Sabbata, Greta Tuckute, Zeming Chen, Martin Schrimpf, Antoine Bosselut

The Mixture of Cognitive Reasoners (MICRO) framework introduces brain-like functional specialization into Large Language Models, aligning expert modules with human cognitive networks such as language, logic, social, and world knowledge. This architecture enhances interpretability, controllability, and human behavioral alignment while maintaining competitive performance on reasoning benchmarks.

Key insights

• Robust functional specialization, mirroring human cognitive networks, can be effectively induced and maintained in LLMs through a targeted training curriculum and a full-block modular architecture.
• This brain-like modularity provides enhanced interpretability through observable token routing patterns and enables causal analysis via expert ablations, revealing a hierarchical processing where early layers handle language grounding while deeper layers delegate to specialized experts.
• Incorporating explicit cognitive inductive biases into LLM design improves alignment with human behavioral patterns and offers a computational framework for formulating testable hypotheses about human brain function.

Summary by alphaXiv.