Reproduce “C-Planning: An Automatic Curriculum for Learning Goal-Reaching Tasks” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tianjun Zhang, Benjamin Eysenbach, Ruslan Salakhutdinov, Sergey Levine, Joseph E. Gonzalez

Researchers at UC Berkeley and Carnegie Mellon University developed C-Planning, an algorithm that trains goal-conditioned reinforcement learning agents to solve long-horizon tasks by automatically generating a curriculum of intermediate waypoints during training. This approach enables superior performance on complex manipulation and navigation challenges while maintaining fast execution at test time.

Key insights

  • Integrating planning directly into the training loop, rather than as a separate test-time step, allows the policy to 'internalize' planning, leading to better learning and efficient deployment.
  • Framing goal-reaching as a latent variable problem and deriving the waypoint distribution through an EM lower bound provides a robust theoretical foundation for automated curriculum generation.
  • Modifying the data collection process with generated waypoints is more effective for policy improvement than solely modifying the RL objective function.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.