Reproduce “Break-It-Fix-It: Unsupervised Learning for Program Repair” on OpenResearch

About this paper

Michihiro Yasunaga, Percy Liang

The Break-It-Fix-It (BIFI) framework, developed at Stanford University, proposes an unsupervised learning approach for automated program repair that bridges the gap between synthetic and real error distributions. It iteratively refines both an error 'breaker' and a 'fixer' using an external 'critic' like a compiler, achieving 90.5% repair accuracy on Python AST errors and 71.7% on C compiler errors, setting new state-of-the-art results without relying on manually labeled bug-fix pairs.

Key insights

• Leveraging an external 'critic' (like a compiler or parser) to verify the correctness of generated code is crucial for filtering noisy pseudo-labels in unsupervised training.
• Iteratively refining both a 'breaker' (error generator) and a 'fixer' (error repairer) allows the system to learn the true distribution of real-world programming errors from unlabeled data.
• Training the fixer directly on successfully self-repaired real bad examples significantly boosts performance, and augmenting this with realistic errors generated by the learned breaker further enhances results.

Summary by alphaXiv.