Reproduce “EmbedGym: Letting Coding Agents Develop and Debug Embedded Firmware Without Physical Hardware” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Minghui Zhao, Yiming Li, Sophie Zhu, Maxwell Khor, Lauren Ouyang, Hai "Helen" Li, Yiran Chen, Tingjun Chen

The authors introduce EmbedGym, a system that lets coding agents test embedded firmware against virtual hardware. It runs compiled binaries with modeled peripherals and lets agents change inputs, trigger events, and inspect hardware state. Across 598 tasks, full interaction raised pass rate to 89.1%; 19.9% of compiled firmware failed assertions with compile-only feedback. Physical-board agreement was 59/60 in sampled firmware.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.