Reproduce “Wizard: Manipulate Any World in Real Time via Agentic Control” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuzhe Wu, Zijun Lin, Yeying Jin
Wizard combines agent-managed world structure and rules with video rendering. It converts scene images into approximate traversable geometry and character references into rigged skeletons; an agent updates these controls while a rule-based controller tracks combat state. Skeleton-derived masks bind references to characters. The authors report 24 FPS and show selected unseen-reference examples without fine-tuning, not broad generalization.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.