Reproduce “MARS: Modular Agent with Reflective Search for Automated AI Research” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiefeng Chen, Bhavana Dalvi Mishra, Jaehyun Nam, Rui Meng, Tomas Pfister, Jinsung Yoon
MARS organizes machine-learning engineering as an evolving repository, using modular code, alternative solutions, and lessons from prior changes to guide later attempts. In MLE-Bench, with Gemini-3-Pro-Preview, it achieved a 56.0 ± 1.5% Any Medal Rate versus 37.8 ± 2.5% for AIRA-dojo in a controlled 24-hour, single-A100 setup. The evidence is limited to MLE-Bench, not open-ended scientific discovery.
Key insights
- Integrating computational budget awareness directly into search heuristics enables the discovery of both high-performing and resource-efficient solutions within practical constraints.
- Adopting a modular, repository-level software architecture significantly enhances an AI agent's ability to manage complex ML codebases, promoting robustness, maintainability, and testability.
- Comparative reflection, by analyzing specific changes between solution versions, effectively resolves the credit assignment problem, distilling precise causal factors for performance shifts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.