Reproduce “Randomized Prior Functions for Deep Reinforcement Learning” on OpenResearch

About this paper

Ian Osband, John Aslanides, Albin Cassirer

Randomized prior functions are integrated into ensemble-based deep reinforcement learning to improve exploration efficiency. This approach enables agents to effectively learn in sparse-reward environments by providing intrinsic motivation, yielding enhanced learning scalability and performance across several challenging tasks.

Key insights

• Injecting prior beliefs into the output function space using randomized prior networks is more effective for deep neural networks than applying regularization in the high-dimensional weight space.
• The additive prior mechanism inherently provides intrinsic motivation, allowing unseen states to be estimated with non-zero values, which drives deep exploration even in environments with zero or negative observed rewards.
• The proposed method helps to ensure proper posterior concentration and propagate multi-step epistemic uncertainty, addressing critical limitations of other uncertainty quantification approaches in sequential decision-making.

Summary by alphaXiv.