Reproduce “RLBenchNet: The Right Network for the Right Reinforcement Learning Task” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ivan Smirnov, Shangding Gu
The paper systematically benchmarks various neural network architectures, including MLPs, LSTMs, Transformers, and Mamba models, within a Proximal Policy Optimization framework across diverse reinforcement learning tasks. It demonstrates that Mamba models achieve up to 4.5x faster training throughput and 8.1x less GPU memory usage compared to LSTMs and Transformers, while often delivering comparable or superior performance, particularly in memory-intensive environments.
Key insights
- Increased architectural complexity does not always translate to superior performance; simpler or more efficient architectures like MLPs and Mamba-2 are often preferable depending on task characteristics.
- Mamba models, particularly Mamba-2, offer a unique combination of high computational efficiency (throughput, memory usage) and competitive performance, making them suitable for resource-constrained or long-horizon tasks.
- Architectural selection should be guided by task requirements, with MLPs for Markovian dynamics, Mamba-2 for temporal dependencies, and Transformers reserved for extremely memory-intensive challenges.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.