Reproduce “Beyond Gemini-3-Pro: Revisiting LLM Routing and Aggregation at Scale” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shengji Tang, Weihao Lin, Peng Ye, Jingqi Ye, Hao Li, Yiqun Zhang, Xiaosong Wang, Bo Zhang and 4 more

A framework named JiSi leverages an orchestrated collaboration of open-source Large Language Models (LLMs) to achieve an average performance gain of 1.15% over Gemini-3-Pro on nine benchmarks. This system also significantly reduces operational costs by 53.23% compared to Gemini-3-Pro, demonstrating a scalable alternative to monolithic LLM architectures.

Key insights

  • Collective intelligence derived from well-orchestrated open-source LLMs can surpass the performance of leading monolithic closed-source models.
  • Integrating LLM-generated response embeddings and reasoning costs significantly enhances routing accuracy by better capturing deep semantics and task difficulty.
  • Dynamic adaptation in aggregator selection and the decision to route versus aggregate is crucial for optimizing performance and resource efficiency across diverse tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.