Reproduce “Carbon-Aware Routing for Function Calling in Edge-Cloud LLM Systems” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aikaterini Maria Panteleaki, Varatheepan Paramanayakam, Spyros Tragoudas, Iraklis Anagnostopoulos
Carbon-aware routing assigns each function-calling request to an edge or cloud LLM tier by jointly estimating execution success, power use, latency, and time-varying grid carbon intensity. Across BFCL V2 and GeoEngine, the system preserves performance close to cloud-only inference while reducing emissions by roughly sixfold on average for BFCL V2, up to eightfold in one configuration, and 1.39-fold on average for the more difficult GeoEngine tasks.
Key insights
- Function-calling workloads benefit from selective edge execution: simpler BFCL V2 requests can be handled locally, whereas difficult sequential GeoEngine calls more often require cloud models.
- Carbon-aware routing differs from fixed model allocation because it accounts for both query difficulty and changing electricity conditions at the execution location.
- Reducing emissions does not merely result from accepting more incorrect answers; the system also lowers carbon per successful query relative to comparable routing strategies.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.