Reproduce “CP-Agent: A Calibrated Risk-Controlled Agent for Feedback-Driven Competitive Programming” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Peisong Wang, Bowen Liu, Zehua Li, Yuyao Wang, Zhiwei Ma, Yuhan Li, Jia Li

A training-free LLM agent, CP-Agent, developed at The Hong Kong University of Science and Technology (Guangzhou), enhances competitive programming performance by integrating a calibrated risk-control framework with execution feedback. It achieved substantial Pass@1 increases, such as +22.7% for DeepSeek-V3.2-Chat on LiveCodeBench Pro, while operating on the cost-accuracy efficiency frontier.

Key insights

  • A principled, calibrated risk-control framework can effectively enable training-free LLM agents to significantly improve competitive programming performance by systematically leveraging execution feedback.
  • Specific mechanisms, such as dual-granularity verification for local hypothesis testing, augmented test generation for evidence acquisition, and experience-driven memory retrieval for context, demonstrably impact distinct aspects of the problem-solving process.
  • Leveraging a frozen, experience-driven memory snapshot not only boosts performance but can also reduce overall computational cost by guiding the agent towards more efficient problem-solving trajectories.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.