Reproduce “GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models” on OpenResearch

About this paper

GLM-4.5 Team, Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang and 163 more

GLM-4.5 combines reasoning, tool use, and coding in one open-weight model. The authors train reasoning, agent-work, and chat specialists, then distill their outputs into a model with thinking and direct response modes. On the authors’ 12-benchmark suite, GLM-4.5 scores 63.2 and ranks third; GLM-4.5-Air scores 59.8 and ranks sixth. These aggregate results apply only to that suite.

Key insights

• The MoE architecture, specifically prioritizing depth over width and using more attention heads, enhances reasoning capacity and parameter efficiency, allowing competitive performance with fewer activated parameters.
• A multi-stage training approach, including specialized 'mid-training' for domain-specific knowledge and iterative expert model post-training, is crucial for unifying and balancing diverse ARC capabilities.
• Advanced Reinforcement Learning techniques, such as difficulty-based curriculum learning, single-stage long-context RL, and iterative self-distillation, effectively refine and boost the model's agentic, reasoning, and general abilities.

Summary by alphaXiv.