Reproduce “Embedding Geometry Shapes LLM Routing: Tracing Failure Modes and Improving Generalization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Leonardo Torti, Emanuele Rimoldi, Tomer Galanti, Tomaso Poggio, Pierfrancesco Beneventano
The authors evaluate language-model routing across held-out task families using a frozen language model’s layer-21 hidden representation instead of semantic embeddings, with kNN-100 and other simple routers. On RouterBench, kNN-100 mean utility across 13 cost penalties rises from 0.6615 to 0.7007. This result covers the tested penalty grid and families; at zero cost penalty, no evaluated router beats the best-single-model baseline, and generalization beyond this encoder pair is unknown.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.