Reproduce “CONCUR: A Framework for Continual Constrained and Unconstrained Routing” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Peter Baile Chen, Weiyue Li, Dan Roth, Michael Cafarella, Samuel Madden, Jacob Andreas

CONCUR is a modular framework designed for continually routing AI tasks to various computational strategies, including large language models and decoding methods, in both constrained and unconstrained settings. It achieved superior end-to-end performance with higher accuracy and reduced inference costs across diverse benchmarks, particularly demonstrating efficient adaptability to new strategies.

Key insights

  • A modular predictor architecture enables seamless integration of new computation strategies with minimal retraining, greatly reducing overhead for continual routing and preventing catastrophic forgetting.
  • Leveraging rich, complementary representations (general-purpose and task-specific) is crucial for capturing complex task-strategy interactions, leading to more informed and optimal routing decisions.
  • Global optimization for constrained routing, distributing a total budget across multiple tasks using dynamic programming, significantly outperforms local, independent task budgeting in maximizing overall accuracy.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.