Reproduce “Hyperbolic Deep Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Edoardo Cetin, Benjamin Chamberlain, Michael Bronstein, Jonathan J Hunt
Cetin et al. integrate hyperbolic geometry into deep reinforcement learning representations, leading to substantial improvements in generalization and sample efficiency on Procgen and Atari 100K benchmarks. Their method improves mean and median normalized scores by 36% on Procgen and nearly doubles human-normalized mean scores on Atari 100K by effectively capturing the hierarchical structure of sequential decision-making.
Key insights
- Markov Decision Processes (MDPs) inherently possess hierarchical, tree-like structures that are well-suited for representation in hyperbolic spaces.
- Empirical analysis showed that learned Euclidean representations with lower δ-hyperbolicity (more tree-like) correlate with improved agent performance and generalization.
- Training deep reinforcement learning agents with hyperbolic representations necessitates specific stabilization techniques, addressed by the proposed Spectrally-Regularized Hyperbolic Mappings (S-RYM).
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.