Reproduce “GenesisFunc: Multi-Agent Data Generation for Accurate and Generalizable Function-Calling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hao-Xiang Xu, Chong Deng, Jiaqing Liu, Wen Wang, Qian Chen, Lujia Bao, Xiangang Li, Zhen-Hua Ling
Researchers at Alibaba Group developed GENESISFUNC, an automated, multi-agent pipeline to synthesize high-quality, diverse function-calling training data for Large Language Models. This pipeline enabled GENESISFUNC-8B to achieve 93.31% accuracy on the BFCL Non-Live split and 83.78% on the Live split, outperforming prior open-source models by relative gains of 2.5% and 3.8%, respectively, and demonstrating strong generalization on out-of-domain benchmarks.
Key insights
- A multi-agent framework, particularly with dedicated Memory and Judge agents, is crucial for generating diverse and high-quality synthetic function-calling data while maintaining semantic coverage and quality control.
- Systematic, multi-stage evaluation, combining rule-based, AI-based, and targeted human validation, is essential to ensure the accuracy and practical usability of synthetic training data for LLMs.
- Employing a dynamic slot-selection strategy during data generation, where subsets of optional parameters are randomly instantiated, significantly enhances the robustness and generalization capabilities of trained function-calling models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.