Reproduce “Solving Minimum-Cost Reach Avoid using Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Oswin So, Cheng Ge, Chuchu Fan

An MIT-developed reinforcement learning framework, RC-PPO, directly solves the minimum-cost reach-avoid problem by reformulating it as an augmented reachability task. It demonstrates superior performance on benchmark control problems, achieving up to 57% lower cumulative costs while maintaining high goal-reaching and safety rates compared to existing methods.

Key insights

  • The minimum-cost reach-avoid problem can be precisely transformed into an augmented reachability problem, enabling a direct optimization approach without heuristic surrogate objectives.
  • Conditioning the policy and value function on the initial cost budget 'z_0' allows a single neural network to learn policies effective across a range of cost constraints.
  • Standard Constrained Markov Decision Processes (CMDPs) and weighted reward formulations are fundamentally limited and cannot consistently achieve true optimal solutions for the minimum-cost reach-avoid problem.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.