Reproduce “The Imitation Game: When LLMs Learn to Reason Like Programs via Code-Centric Reasoning Data Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jinyang Zhang, Weibin Liao, Keqin Bao, Sihang Li, Shaobo Wang, Muyang Ye, Hongxin Ding, Yue Fang and 5 more
MIMIC turns accepted programs into training supervision for natural-language reasoning by combining code-derived problem kernels, narrative rewrites, execution-guided test cases, and verified intermediate traces. Across Qwen3 and DeepSeek-R1-Distill models, the resulting data improves general and mathematical reasoning, with Qwen3-8B gains from 22.0% to 40.0% on MazeBench and from 26.2% to 39.5% on StringBench; narrative reformulation also reduces code-template behavior on Codeforces-style prompts.
Key insights
- Preserving program logic while changing the surrounding story improves transfer: raw-Codeforces SFT reduced Qwen3-4B general and mathematical averages by 16.5 and 21.6 points relative to MIMIC SFT.
- Execution traces provide a scalable source of process supervision for deterministic tasks, covering branches, loop behavior, and state changes rather than only final answers.
- Narrative reformulation suppresses surface-level code recall: Qwen3-32B’s code-writing rate fell from 38.7% to 0.7%, while accuracy rose from 23.8% to 42.2%.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.