Reproduce “Random Latent Exploration for Deep Reinforcement Learning” on OpenResearch

About this paper

Srinath Mahankali, Zhang-Wei Hong, Ayush Sekhari, Alexander Rakhlin, Pulkit Agrawal

Researchers at MIT and the Improbable AI Lab developed Random Latent Exploration (RLE), a method that guides deep reinforcement learning agents to explore diverse states by conditioning their policies on randomly sampled latent goals. RLE achieves superior average performance across various Atari and Isaac Gym benchmarks, combining the simplicity of noise-based exploration with the deep exploration capabilities of complex bonus-based approaches.

Key insights

• Encouraging diverse exploration through randomly sampled latent goals, which define different reward functions, is an effective strategy for achieving deep state space coverage.
• Explicitly conditioning the policy and value networks on these latent goals is crucial, enabling the agent to learn goal-conditioned behaviors rather than treating the random rewards as mere noise.
• It is possible to achieve deep, diverse exploration with a simple, plug-in method that avoids the computational overhead and complexity associated with traditional bonus-based approaches.

Summary by alphaXiv.