Reproduce “Workflow-R1: Group Sub-sequence Policy Optimization for Multi-turn Workflow Construction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mingze Kong, Zikun Qu, Zhongquan Zhou, Pengyu Liang, Xiang Li, Zhiwei Shang, Zhi Hong, Kaiyu Huang and 2 more
Workflow-R1 introduces a dynamic, multi-turn approach to LLM-based agent workflow construction, utilizing Group Sub-sequence Policy Optimization (GSsPO) to learn efficient reasoning paths from scratch. This framework achieves state-of-the-art performance on seven QA benchmarks and fosters advanced reflective and self-correction capabilities in agents.
Key insights
- Reformulating workflow construction as a dynamic, multi-turn, natural language-based process enables agents to adapt reasoning and operator selection based on real-time observations, surpassing static approaches.
- The 'sub-sequence' (atomic Think-Action pair) represents an optimal optimization granularity for multi-turn agentic reasoning, effectively bridging the gap between fine-grained token-level and coarse-grained sequence-level RL.
- Training with dynamic interaction and appropriate reward design fosters the emergence of deep reflective capabilities, including self-correction and adaptive strategy refinement, in LLM agents.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.