Reproduce “Massively Scalable Inverse Reinforcement Learning in Google Maps” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Matt Barnes, Matthew Abueg, Oliver F. Lange, Matt Deeds, Jason Trader, Denali Molitor, Markus Wulfmeier, Shawn O'Banion
Google researchers developed a massively scalable Inverse Reinforcement Learning framework, including the Receding Horizon Inverse Planning (RHIP) algorithm, to infer user preferences for route recommendation in Google Maps. This approach achieved a 15.9% increase in accuracy for driving and 24.1% for two-wheelers compared to baselines on a planetary-scale road network.
Key insights
- An intermediate receding horizon (e.g., H=10) in RHIP can achieve superior accuracy and faster training compared to an infinite horizon (MaxEnt), indicating a more efficient trade-off for modeling human-like decision-making.
- Localized routing preferences are prevalent, as shown by the effectiveness of the Sparse Mixture of Experts and the performance drop when experts are evaluated outside their training regions.
- High-parameter SparseLin reward models can implicitly identify and correct underlying data quality errors in the road network by assigning specific edge-level rewards, like for mislabeled private roads frequently used by users.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.