Reproduce “An Empirical Study of World Model Quantization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhongqian Fu, Tianyi Zhao, Kai Han, Hang Zhou, Xinghao Chen, Yunhe Wang
Researchers from Huawei Noah's Ark Lab empirically studied Post-Training Quantization (PTQ) techniques on DINO-WM, a Transformer-based world model for visual planning. Their findings detail how various quantization strategies, bit-widths, and granularities impact the model's performance and stability in iterative planning tasks, highlighting the unique challenges of temporal error accumulation.
Key insights
- Group-wise weight quantization improves stability at 4-bit precision, allowing partial performance recovery, but fails under more extreme precision reductions like 3-bit.
- Per-token activation quantization offers limited and inconsistent gains over per-tensor quantization for world model planning, and can sometimes introduce additional instability.
- World models exhibit significantly higher sensitivity to quantization in the encoder module, which determines the latent representation, compared to the predictor module.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.