Reproduce “Simple random search provides a competitive approach to reinforcement learning” on OpenResearch

About this paper

Horia Mania, Aurelia Guy, Benjamin Recht

Augmented Random Search (ARS), a simple derivative-free optimization algorithm utilizing linear policies, achieves competitive or superior sample efficiency compared to complex state-of-the-art reinforcement learning methods on standard MuJoCo locomotion benchmarks. The work also highlights significant performance variance in RL algorithms through rigorous evaluation with 100 random seeds, advocating for more robust evaluation methodologies.

Key insights

• Simple linear policies are sufficient for achieving high performance on various MuJoCo locomotion tasks, challenging the necessity of complex neural network architectures.
• Reinforcement learning algorithms, including ARS, exhibit substantial performance variance across random seeds and hyperparameter configurations, underscoring the need for more rigorous evaluation.
• Rigorous evaluation practices, using a high number of random seeds (e.g., 100), are crucial for accurately assessing and comparing RL algorithm performance and understanding their true variability.

Summary by alphaXiv.