Reproduce “AutoKaggle: A Multi-Agent Framework for Autonomous Data Science Competitions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ziming Li, Qianbo Zang, David Ma, Jiawei Guo, Tuney Zheng, Minghao Liu, Xinyao Niu, Yue Wang and 6 more
AutoKaggle organizes tabular Kaggle competition work into six phases, using specialized agents, executable code, and phase-specific unit tests to plan, implement, review, and repair each stage. Across eight selected competitions and five trials, its GPT-4o setting averaged 0.83 valid submissions and a 0.821 comprehensive score, versus 0.58 and 0.641 for AIDE, with results limited to this benchmark.
Key insights
- Unit tests are critically important for ensuring logical correctness in data science tasks, with their absence causing a significant drop in completion rates (to 0.10).
- The ML Tools Library primarily enhances the stability and completion rate of the framework by offloading complex domain-specific code generation from LLMs.
- The structured multi-agent approach, coupled with iterative debugging, allows the system to effectively manage complex end-to-end data science tasks and generalize robustly to novel problems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.