Reproduce “GrndCtrl: Grounding World Models via Self-Supervised Reward Alignment” on OpenResearch

About this paper

Haoyang He, Jay Patrikar, Dong-Ki Kim, Max Smith, Daniel McGann, Ali-akbar Agha-mohammadi, Shayegan Omidshafiei, Sebastian Scherer

Researchers at Carnegie Mellon University and FieldAI developed GrndCtrl, a self-supervised framework that grounds large-scale video world models by aligning their learned dynamics with verifiable physical geometry and temporal invariants using reinforcement learning. This approach substantially reduces translation and rotation errors by up to 45% and 77% respectively, particularly improving stability and generalization in counterfactual and unseen environments.

Key insights

• Explicit pose-based feedback (translation and rotation rewards) significantly enhances motion consistency and stabilizes rollouts, particularly in counterfactual scenarios requiring extrapolation to novel motion patterns.
• Combining all verifiable rewards (geometric and perceptual) yields the most balanced and robust performance, recovering accuracy benefits while maintaining local stability and visual coherence.
• The stability of GRPO training depends on sufficient reward variance across stochastic rollouts and benefits from computing likelihood ratios only over the early, critical diffusion timesteps to focus on primary scene content denoising.

Summary by alphaXiv.