Reproduce “Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Control” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Mingyang Chen, Shengdong Chen, Xiaoxiao Fu, Bosheng Gong, Haoyuan Guo, Bowen Li, Jiawen Li, Kejun Li and 8 more

Zing-0.5 is a 5-billion-parameter autoregressive world model that combines continuous keyboard navigation with online text instructions in a single video stream. Using segment-level event supervision, four-step distillation, cached attention, and a streaming runtime, it reaches 24.63 frames per second at 832×480 on RTX 5090 hardware, scores 81.0 overall on the 158-case WBench Navigation split, and is estimated to cost about $0.009 per stream-minute.

Key insights

  • Keyboard inputs provide continuous navigation and viewpoint control, while interval-specific text prompts can alter characters, objects, and environmental events during the same ongoing session.
  • Training on generated histories, perturbed context, and long clips is used to improve robustness against autoregressive degradation rather than relying only on clean teacher-forced context.
  • The system demonstrates playable visual interaction but lacks an explicit world state or rule-based memory, so instructed changes may not persist reliably after leaving the visible context.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.