Reproduce “Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sicheng Mo, Yuheng Li, Ziyang Leng, Krishna Kumar Singh, Bolei Zhou
WorldWeaver, a streaming multi-agent autoregressive diffusion model from researchers at UCLA and Adobe Research, introduces "world state registers" to explicitly maintain a shared, dynamically updated world state across agents. This approach generates visually and logically consistent multi-agent videos, achieving an aggregate WorldScore of 105.1, which improves upon baselines and shows notable gains in VLM accuracy for state-sensitive categories such as grounding and consistency.
Key insights
- Explicit world state modeling through persistent registers substantially improves logical and geometric consistency in multi-agent video generation, outperforming models reliant on implicit memory.
- The effectiveness of world state registers is significantly enhanced by multi-faceted supervision signals, including local agent dynamics, global scene geometry, and high-level semantic descriptions.
- A Mixture-of-Transformers architecture helps to balance competing objectives when registers encode rich semantic information, stabilizing learning and improving generation quality, particularly under explicit supervision.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.