Reproduce “Q-Learning in enormous action spaces via amortized approximate maximization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tom Van de Wiele, David Warde-Farley, Andriy Mnih, Volodymyr Mnih
AQL learns a state-dependent proposal distribution to suggest candidate actions, then selects the highest-valued candidate rather than scoring every possible action. On two DeepMind Lab exploration tasks, AQL with 3,528 actions achieved the best final mean episode return among compared methods; exact Q-learning took about ten times longer in wall-clock time. The approximate search does not guarantee the global maximum.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.