Reproduce “OptLLM: Optimal Assignment of Queries to Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yueyue Liu, Hongyu Zhang, Yuantian Miao, Van-Hoang Le, Zhiqiang Li
The OptLLM framework optimizes large language model usage by intelligently assigning queries to specific LLMs, balancing computational cost and accuracy. It achieved cost reductions of 2.40% to 49.18% for equivalent accuracy compared to using a single best LLM, and up to 69.05% accuracy improvements over other multi-objective optimization methods.
Key insights
- Formulating query assignment as a multi-objective optimization problem allows balancing cost and accuracy in LLM usage.
- A robust-aware prediction component, based on bootstrap sampling and weighted aggregation, estimates LLM success probabilities for individual queries.
- A heuristic search algorithm optimizes query distribution, generating Pareto-optimal solutions for cost-accuracy trade-offs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.