Reproduce “Bayesian control for coding agents” on OpenResearch

About this paper

Theodore Papamarkou, Vladislav Smirnov, Viktor Mazanov, Artem Vazhentsev, Preslav Nakov, Timothy Baldwin, Artem Shelmanov

This research develops a cost-sensitive Bayesian decision-theoretic framework for orchestrating Large Language Model (LLM) coding agents, formalizing code generation as a sequential hypothesis testing problem. The proposed controllers dynamically balance tool costs and information gain, demonstrating utility improvements of approximately 67 units over baselines in high-cost verification scenarios and providing an effective uncertainty quantification score with an 86.6% Prediction Rejection Ratio.

Key insights

• The optimal orchestration strategy for coding agents is highly dependent on the initial success probability and the verifier cost-to-reward ratio, leading to distinct regimes where Bayesian control, a public-test gate, or blind verification is optimal.
• Bayesian controllers effectively aggregate noisy signals from multiple imperfect critics to maintain a robust posterior belief, enabling superior cost-adjusted performance when verification is expensive and critics are informative but fallible.
• The posterior belief state derived from the Bayesian framework provides an interpretable and effective uncertainty quantification score for generated code, outperforming simpler token-probability baselines.

Summary by alphaXiv.