Reproduce “SimCRAFT: Distilling Remote Sensing Agents via Synthetic Trajectories and Contextual Retrieval-Augmented Fine-Tuning” on OpenResearch

About this paper

Haoran Wang, Jing Yao, Xu Yang, Zeqing Wang, Yang Zhang, Pedram Ghamisi, Zhengchao Chen

The SimCRAFT framework distills sophisticated remote sensing agent orchestration into a compact, open-source 7-billion-parameter model using multi-agent data synthesis and noise-robust contextual fine-tuning. This approach achieved an 83.2% Planning Success Rate, rivaling proprietary frontier models and outperforming larger open-source LLMs, while generalizing effectively to unseen tasks and tool spaces.

Key insights

• High-quality, physically consistent RS tool-use trajectories for complex multi-step tasks can be synthesized at scale using a multi-agent system coordinated by a shared state machine and validated by a Mock Execution Engine.
• Explicitly training compact LLMs to be robust to noisy contextual retrieval during fine-tuning (CRAFT) enables them to extract transferable procedural structures, leading to superior generalization compared to simple supervised fine-tuning or inference-time RAG.
• Distilling sophisticated domain-specific agentic capabilities into 7B-scale open-source models is achievable, allowing them to rival or even surpass much larger proprietary models in specific task metrics and be suitable for edge deployment.

Summary by alphaXiv.