Reproduce “Learning World Models with Identifiable Factorization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yu-Ren Liu, Biwei Huang, Zhengmao Zhu, Honglong Tian, Mingming Gong, Yang Yu, Kun Zhang

The IFactor framework introduces a novel categorization of latent state variables in reinforcement learning into four identifiable types (controllable/reward-relevant, uncontrollable/reward-relevant, controllable/reward-irrelevant, and uncontrollable/reward-irrelevant). It provides theoretical guarantees for block-wise identifiability of these categories in nonlinear settings and demonstrates that leveraging only reward-relevant states leads to more robust and sample-efficient policy learning in complex environments.

Key insights

  • A principled categorization of latent variables into four types based on controllability and reward relevance offers a structured and transparent understanding of complex environmental states.
  • Block-wise identifiability of these disentangled latent categories is achievable in general nonlinear settings, providing a robust theoretical foundation for causal representation learning in RL.
  • For optimal and robust policy learning, only the reward-relevant latent variables (s_r_t) are necessary and sufficient, enabling effective filtering of distractors and improving sample efficiency.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.