Reproduce “Maestro: Reinforcement Learning to Orchestrate Hierarchical Model-Skill Ensembles” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jinyang Wu, Guocheng Zhai, Ruihan Jin, Yuhao Shen, Zhengxi Lu, Fan Zhang, Haoran Luo, Zheng Lian and 2 more
MAESTRO introduces an RL-driven orchestrator that dynamically composes ensembles of frozen expert models and a two-tier skill library to tackle heterogeneous multimodal tasks. This framework achieved an average accuracy of 70.1% across ten multimodal benchmarks, outperforming GPT-5 (69.3%) and Gemini-2.5-Pro (68.7%), while demonstrating extensibility to unseen experts and skills without retraining.
Key insights
- Decoupling the coordination logic into a lightweight, RL-trained orchestrator allows it to manage powerful, frozen expert models and skills effectively, rather than relying on a single monolithic model.
- Dynamic, learned orchestration enables superior performance and generalizability by strategically routing tasks to the most appropriate combination of specialized models and fine-grained skills for each step.
- A combined reward system incorporating both task-specific accuracy and structural adherence is crucial for training a stable and high-performing orchestrator in complex sequential decision-making environments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.