Reproduce “Yet Even Less Is Even Better For Agentic, Reasoning, and Coding LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
CodeArts Model Team, Yang Ye, Jingyuan Tan, Tianyue Jiang, Ruizhe Ye, Qiankun He, Jiarui Yang, Jian Dong and 19 more
The CodeArts Model Team introduced an end-to-end framework, SandForge and STITCH, that enhances the agentic capabilities of large language models for software engineering tasks across multiple languages and scales by training on a reduced volume of high-quality, semantically-rich trajectories. This approach led to consistent performance improvements on benchmarks like SWE-bench, Multi-SWE-Bench (Java), and a custom HarmonyOS benchmark, demonstrating more efficient learning and better generalization.
Key insights
- The "Less-Is-More" paradigm, emphasizing quality over quantity of data, is effective for complex agentic tasks involving coding, reasoning, and tool use, not just pure mathematical reasoning.
- A multi-stage, fine-grained data curation process that combines statistical filtering with semantic analysis via LLM-as-a-judge is crucial for generating high-signal training datasets.
- The methodology is language-agnostic and scalable, demonstrating effectiveness across various model sizes and in low-resource language environments like ArkTS, promoting robust and generalizable agentic capabilities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.