Reproduce “Exploration via Elliptical Episodic Bonuses” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Mikael Henaff, Roberta Raileanu, Minqi Jiang, Tim Rocktäschel

Researchers at Meta AI and University College London developed Exploration via Elliptical Episodic Bonuses (E3B), an approach that generalizes count-based exploration to continuous state representations for Contextual Markov Decision Processes. E3B achieved superior performance across 16 MiniHack tasks and enabled agents to explore substantially larger areas in photorealistic Habitat environments compared to existing methods.

Key insights

  • Count-based episodic bonuses are critical for the performance of existing CMDP exploration methods but are fragile to unique states caused by noise or high-dimensionality.
  • Automatically learning state representations via an inverse dynamics model helps agents focus on controllable environmental aspects, making exploration robust against irrelevant observational noise.
  • Generalizing episodic novelty from discrete counts to continuous, statistically-modeled regions of an embedding space enables effective exploration in complex, realistic CMDPs.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.