Reproduce “Finding the Time to Think: Learning Planning Budgets in Real-Time RL” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aneesh Muppidi, Firas Darwish, Dylan Cope, João F. Henriques, Jakob Nicolaus Foerster
Researchers developed a method for reinforcement learning agents to learn dynamic planning budgets in real-time environments where computation incurs a cost, outperforming fixed-budget strategies across five diverse games. This approach also facilitated successful transfer of simulation-trained policies to asynchronous multi-GPU hardware deployments without modification.
Key insights
- The cost of computation can be effectively integrated into the environment dynamics by formalizing variable-delay real-time reinforcement learning.
- A lightweight gating policy can learn to adaptively allocate planning budgets for a complex, pre-trained planner without incurring a significant meta-reasoning overhead.
- Learned adaptive planning strategies dynamically prioritize deeper deliberation in specific state contexts, such as high-risk scenarios or when spatial constraints are tight.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.