Reproduce “Cycle-Consistent World Models for Domain Independent Latent Imagination” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sidney Bender, Tim Joseph, Marius Zoellner
Cycle-Consistent World Models (CCWM) learn shared latent dynamics for different observation modalities, such as RGB and semantic segmentation images, without requiring aligned data. The method enables domain-independent latent imagination and improves reward and next-state prediction accuracy in autonomous driving simulations, surpassing existing domain adaptation techniques.
Key insights
- A shared-latent space, enforced by cycle-consistency and adversarial training, allows a world model to learn dynamics from one observation modality and generalize to another without paired cross-domain data.
- A sufficiently large latent space (e.g., 8x8) is crucial for effective representation and learning in complex scenes with multiple objects, providing better inductive biases.
- Training an end-to-end model-based RL system with integrated domain adaptation mechanisms can outperform methods that rely on separate translation steps or simple domain randomization, leading to more robust sequential predictions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.