Reproduce “A Control System, a Dataset, and a Recipe for Making Frozen LLM Agents Learn a Domain” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Debjyoti Paul
Researchers developed a control system for optimizing frozen LLM agent harnesses using a constrained, human-legible action space, finding that a DSPy-optimized static baseline consistently matched or surpassed online adaptive controllers across multiple domains while incurring lower costs due to the sample efficiency gap of online methods.
Key insights
- Online adaptive controllers (ϵ-greedy contextual bandit, REINFORCE) exhibited significant sample-efficiency challenges in the large discrete action space, failing to converge to optimal harness configurations within practical episode budgets.
- DSPy's metric-guided BootstrapFewShot method, used for the static baseline, proved highly sample-efficient by proposing plausible few-shot demonstration sets, outperforming online learning in initial phases.
- Harness optimization for production environments benefits from a constrained, auditable action space and a multi-objective reward function, providing a practical alternative to complex, arbitrary code-generating approaches.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.