Reproduce “Causal LLM Routing: End-to-End Regret Minimization from Observational Data” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Asterios Tsiourvas, Wei Sun, Georgia Perakis

Researchers from MIT and IBM Research developed a causal end-to-end framework for dynamically routing queries to Large Language Models (LLMs), learning optimal policies directly from observational data by minimizing decision-making regret. This approach effectively addresses treatment bias and heterogeneous cost preferences, achieving superior utility compared to prior methods on public benchmarks.

Key insights

  • Directly minimizing decision-making regret from observational data, as opposed to decoupled prediction, yields more robust and effective LLM routing policies.
  • Causal inference techniques, such as the doubly robust estimator, are essential for unbiased estimation of counterfactual outcomes when learning LLM routing policies from biased observational data.
  • The optimal LLM selection is piecewise constant with respect to cost sensitivity, enabling efficient and accurate generalization to a continuous range of user preferences through interval-based interpolation.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.