Reproduce “Next-Scale Autoregressive Models for Text-to-Motion Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zhiwei Zheng, Shibo Jin, Lingjie Liu, Mingmin Zhao

Researchers at the University of Pennsylvania developed MoScale, a next-scale autoregressive framework for text-to-motion generation that processes information hierarchically. This approach enhances text-motion alignment and motion quality, achieving state-of-the-art performance with a Top-1 R-Precision of 0.540 and an FID of 0.046 on the HumanML3D dataset, while also supporting zero-shot generalization for motion editing tasks.

Key insights

  • A hierarchical coarse-to-fine causal structure in autoregressive models provides a more effective semantic scaffold for long-range motion structures, overcoming the 'short-horizon shortcut' of previous AR methods.
  • Integrating targeted refinement mechanisms (cross-scale and in-scale) within an autoregressive framework significantly boosts robustness and local fidelity, enabling AR models to perform well even with limited data.
  • Autoregressive models, when designed with next-scale prediction and refinement, can achieve and surpass the performance of diffusion models and masked transformers in text-to-motion generation, demonstrating high training efficiency and versatile zero-shot generalization.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.