Reproduce “EvolveCoder: Evolving Test Cases via Adversarial Verification for Code Reinforcement Learning” on OpenResearch

About this paper

Chi Ruan, Dongfu Jiang, Huaye Zeng, Ping Nie, Wenhu Chen

EvolveCoder introduces a framework that evolves test cases through adversarial verification to improve code reinforcement learning. This method generates the EVOLVECODER-22K dataset with iteratively refined, challenging, and discriminative tests, leading to a 2.4-point average performance improvement on coding benchmarks for a 4B model trained on the final dataset compared to initial training data.

Key insights

• The execution behaviors of diverse candidate solutions directly reveal aspects of program semantics not adequately covered by static test suites, enabling dynamic test refinement.
• Iteratively generating adversarial and discriminative tests can systematically increase the difficulty and discriminative power of test suites while reducing redundancy.
• Dynamically evolving test suites based on observed model behaviors leads to more stable optimization dynamics and consistent performance gains in code generation through reinforcement learning.

Summary by alphaXiv.