Reproduce “Fast Task Inference with Variational Intrinsic Successor Features” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Steven Hansen, Will Dabney, Andre Barreto, Tom Van de Wiele, David Warde-Farley, Volodymyr Mnih
An algorithm named Variational Intrinsic Successor FeatuRes (VISR) was developed by DeepMind, unifying Behavioral Mutual Information (BMI) for unsupervised skill acquisition with Successor Features (SFs) for fast transfer learning. This integration allowed for automatic feature learning and rapid task inference, enabling agents to achieve human-level performance on 12 Atari games and better-than-random scores on 44 games with only 100,000 supervised steps after pre-training.
Key insights
- Behavioral Mutual Information (BMI) maximization can be used to automatically learn the task-relevant features ("φ") that are crucial for Successor Features (SFs).
- By framing the BMI's latent skill space as the SF's linear reward weight space, fast task inference and enhanced generalization can be achieved for intrinsically motivated agents.
- Combining learned features with Generalized Policy Improvement (GPI) allows for robust policy derivation and adaptation to new tasks using minimal supervised data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.