Reproduce “Compiling Agentic Workflows into LLM Weights: Near-Frontier Quality at Two Orders of Magnitude Less Cost” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Simon Dennis, Rivaan Patil, Kevin Shabahang, Hao Guo
Researchers at i14, University of Melbourne, present an empirical study demonstrating that compiling agentic workflows into a smaller LLM's weights, termed "subterranean agents," achieves near-frontier model quality for procedural tasks. This method reduces inference costs by two orders of magnitude compared to existing approaches and maintains flexibility for iterative development.
Key insights
- Embedding persistent procedural structure directly into an LLM's weights offers architectural advantages, such as holistic reasoning, zero routing failures, and unconstrained natural conversational responses, which compensate for smaller model sizes.
- The significant cost reduction stems from both a substantial decrease in per-token inference cost through self-hosting and a reduction in token volume due to constant-size, minimal prompts.
- Compiling workflows into model weights is not inherently rigid; rapid recompile cycles (under an hour on cluster hardware) allow for iterative development and integration into CI/CD pipelines.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.