Reproduce “ICL-Router: In-Context Learned Model Representations for LLM Routing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chenxu Wang, Hao Li, Yiqun Zhang, Linyao Chen, Jianhao Chen, Ping Jian, Peng Ye, Qiaosheng Zhang and 1 more
ICL-Router introduces a method for Large Language Model (LLM) routing that uses in-context learned model representations, achieving an average accuracy of 76.30% on in-distribution tasks and 66.47% on out-of-distribution tasks. The approach allows for scalable integration of new LLMs without requiring retraining of the router, simply by creating new capability profiles.
Key insights
- In-context learned model representations, derived from query-performance pairs, provide a semantically rich and scalable way to characterize LLM capabilities for routing.
- Decoupling the model capability profiling from the routing decision process drastically improves scalability, allowing seamless integration of new LLMs without router retraining.
- A Query Reconstruction Training stage is essential for aligning the semantic spaces of the projector and router, ensuring effective interpretation of query vector representations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.