Reproduce “Experience-Based Heuristic Search: Robust Motion Planning with Deep Q-Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Julian Bernhard, Robert Gieselmann, Klemens Esterle, Alois Knoll
Experience-Based Heuristic Search (EBHS), developed by researchers from fortiss GmbH and TU München, integrates Deep Q-Network (DQN) learned Q-values as heuristics into a Hybrid A* search algorithm for robust and efficient autonomous vehicle motion planning. This method achieved up to a 60% reduction in median planning duration and a drastic decrease in expanded nodes, while consistently providing solutions in scenarios where pure DRL planners failed.
Key insights
- Generating an effective learned heuristic from DRL requires careful selection of hyperparameters, as trade-offs exist between temporal approximation error, network capacity, and the accuracy of the resulting heuristic estimates.
- For the non-holonomic heuristic learning application, a one-step DQN with a smaller network capacity (n=1, 3x300 units) offered the most balanced heuristic accuracy, minimizing total error despite increased variance.
- Integrating a learned Q-function effectively prunes the search space by guiding the algorithm with prior experience, thereby drastically reducing the number of expanded nodes compared to relying solely on analytical heuristics.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.