Reproduce “Hadamax Encoding: Elevating Performance in Model-Free Atari” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jacob E. Kooi, Zhao Yang, Vincent François-Lavet

The Hadamax encoder, developed by researchers at Vrije Universiteit Amsterdam, significantly boosts the performance of pixel-based model-free reinforcement learning agents on the Atari-57 benchmark by approximately 80% over the PQN baseline. This novel architecture, combining max-pooling, Hadamard representations, and GELU activations, allows PQN to surpass Rainbow-DQN's performance much faster while maintaining computational efficiency.

Key insights

  • Significant performance gains in model-free RL are achievable through targeted architectural innovation rather than solely relying on algorithmic changes or increased network depth.
  • Hadamard representations effectively increase the network's capacity and effective rank in deeper layers, supporting stable training even with zero-saturating activation functions like GELU.
  • Max-pooling is crucial for efficient feature compression and selection, while the synergistic combination of max-pooling, Hadamard representations, and GELU is key to the encoder's performance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.