Reproduce “Causal Dynamics Learning for Task-Independent State Abstraction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zizhao Wang, Xuesu Xiao, Zifan Xu, Yuke Zhu, Peter Stone
Researchers at The University of Texas at Austin developed Causal Dynamics Learning (CDL), a framework that integrates causal inference with Model-Based Reinforcement Learning to learn task-independent state abstractions. This approach yields dynamics models with superior generalization to out-of-distribution states and improves sample efficiency for downstream tasks, accurately inferring causal graphs and maintaining prediction accuracy where dense models degrade by 60-90%.
Key insights
- Explicitly learning and leveraging causal dependencies, rather than mere correlations, is critical for building dynamics models that generalize robustly to unseen states.
- State abstractions derived purely from the underlying causal dynamics are inherently task-independent, enabling broad reusability and efficiency across a wide array of future tasks.
- An exploration strategy that targets discrepancies between dense and causal model predictions effectively guides the agent to uncover and refine true causal relationships in the environment.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.