Reproduce “Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sumedh A. Sontakke, Arash Mehrjou, Laurent Itti, Bernhard Schölkopf
Sontakke et al. developed a framework called Causal Curiosity, which enables reinforcement learning agents to autonomously discover and disentangle causal factors in their environment through self-supervised experimentation. This approach leads to improved sample efficiency and robust zero-shot generalization for various downstream tasks.
Key insights
- Intrinsic motivation, when specifically aligned with discovering causal mechanisms, can induce agents to learn interpretable, goal-oriented experimental behaviors that expose underlying environmental properties.
- A hierarchical, interventionist approach allows for the unsupervised construction of a disentangled binary latent space, where each dimension corresponds to an isolated causal factor.
- Conditioning downstream policies on these learned disentangled causal representations significantly enhances an agent's sample efficiency and its ability to generalize zero-shot to entirely unseen environments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.