Reproduce “Solar Open 2 Technical Report” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sungrae Park, Sanghoon Kim, Gyoungjin Gim, Jungho Cho, Hyunwoong Ko, Minbyul Jeong, Minjeong Kim, Keunwoo Choi and 45 more
Solar Open 2 completes a sandboxed FATF compliance task by screening transactions against four rules and producing a regulator-facing report and spreadsheet. Its hybrid attention alternates full-attention layers with cheaper linear-attention layers, and purpose-built agent scenarios support training. On Ko-GDPval, it scores 86.8, versus DeepSeek-V4-Pro’s 86.9; this benchmark result does not establish general workplace reliability.
Key insights
- A hybrid attention stack combining softmax and linear attention, alongside the absence of positional encoding (NoPE), offers a computationally efficient architecture for achieving 1M-token context windows.
- Initializing training with a 'selective weight transfer' from a previous model provides a substantial acceleration (approximately 1.7x faster) in reaching training performance targets during pre-training.
- Multi-teacher On-Policy Distillation (MOPD) can effectively integrate diverse agent specialist skills into a cohesive generalist model by providing dense, token-level supervision.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.