Reproduce “Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ailin Huang, Ang Li, Aobo Kong, Bin Wang, Binxing Jiao, Bo Dong, Bojun Wang, Boyu Chen and 208 more
The StepFun Team developed Step 3.5 Flash, an 11-billion active parameter Mixture-of-Experts model that achieves frontier-level intelligence comparable to leading closed-source LLMs while emphasizing computational efficiency for agentic systems. It shows robust performance across reasoning and agentic benchmarks, making sophisticated AI more deployable.
Key insights
- Frontier-level agentic intelligence can be attained with a relatively low active parameter count (11B) through a co-designed architecture focused on computational efficiency.
- Stable training of large sparse MoE models requires specific diagnostics and mitigation techniques to prevent issues like expert collapse and activation blow-ups.
- Scalable and stable Reinforcement Learning for MoE models in long-horizon reasoning benefits from innovations like Metropolis Independence Sampling-Filtered Policy Optimization (MIS-PO).
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.