Reproduce “QuantVLA: Scale-Calibrated Post-Training Quantization for Vision-Language-Action Models” on OpenResearch

About this paper

Jingxuan Zhang, Yunta Hsieh, Zhongwei Wan, Haokun Lin, Xin Wang, Ziqi Wang, Yingtie Lei, Mi Zhang

QuantVLA, a training-free post-training quantization framework, effectively quantizes Vision-Language-Action (VLA) models, including their sensitive Diffusion Transformer (DiT) action heads. This approach reduces memory footprint by up to 70% while maintaining or slightly improving task success rates on state-of-the-art VLA policies like OpenPI π0.5 and GR00T N1.5.

Key insights

• Quantization-induced scale drifts in VLA models primarily affect the effective attention-logits temperature and residual-stream energy within the sensitive DiT action head, leading to instability.
• Selectively keeping attention projection layers in full precision while quantizing other components is crucial for maintaining performance stability in VLA models with DiT action heads.
• Lightweight, training-free scalar calibrations (ATM and OHB) can effectively counteract quantization-induced scale shifts in attention and residual paths, enabling stable low-bit inference without new operators.

Summary by alphaXiv.