Reproduce “ACC: Compiling Agent Trajectories for Long-Context Training” on OpenResearch

About this paper

Qisheng Su, Zhen Fang, Shiting Huang, Yu Zeng, Yiming Zhao, Kou Shi, Ziao Zhang, Lin Chen and 3 more

Agent Context Compilation (ACC) converts multi-turn agent trajectories into long-context question-answering training data for large language models (LLMs). This data-centric method directly supervises long-range information integration, enabling a 30 billion parameter model to achieve long-context reasoning performance comparable to models with nearly 8 times more active parameters on benchmarks like MRCR and GraphWalks, while preserving general capabilities.

Key insights

• The 'supervision blind spot' in standard agent SFT significantly hinders long-context reasoning by optimizing for local actions rather than global evidence integration.
• Directly supervising the model on compiled contexts, where all evidence is present, effectively forces the LLM to learn long-range dependencies and integrate scattered information.
• Training with diverse agent trajectory types (Search, Software Engineering, SQL) provides complementary benefits, leading to more robust improvements in various long-range dependency tasks.
• ACC training induces adaptive changes in the model's internal mechanisms, including task-specific attention restructuring and expert specialization in Mixture-of-Experts (MoE) architectures.

Summary by alphaXiv.