Reproduce “Approximate Thompson Sampling via Epistemic Neural Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ian Osband, Zheng Wen, Seyed Mohammad Asghari, Vikranth Dwaracherla, Morteza Ibrahimi, Xiuyuan Lu, Benjamin Van Roy
DeepMind researchers developed ENN-DQN, an approach leveraging Epistemic Neural Networks (ENNs) to enable approximate Thompson Sampling in deep reinforcement learning. This method, particularly with the epinet architecture, matches the exploration performance of large ensembles at significantly reduced computational costs, confirming that accurate joint predictive distributions are key for effective deep exploration.
Key insights
- Epistemic Neural Networks, particularly the epinet architecture, provide a computationally efficient method for approximate Thompson Sampling in deep reinforcement learning.
- The quality of *joint predictive distributions* from an ENN is crucial and directly correlates with effective decision-making performance in exploration tasks.
- Traditional marginal uncertainty estimates are insufficient for robust deep exploration in complex environments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.