Reproduce “DLLMQuant: Quantizing Diffusion-based Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chen Xu, Dawei Yang
DLLMQuant provides a post-training quantization framework for Diffusion-based Large Language Models (DLLMs) that addresses the severe accuracy degradation observed with existing methods. It achieves an average accuracy improvement of over 2% across nine NLP tasks and three DLLM architectures, concurrently yielding over 3.2x memory savings and 1.6x inference speedup compared to full-precision models.
Key insights
- The iterative and dynamic nature of DLLM generation, with varying mask ratios and token uncertainties, requires specialized calibration and error compensation beyond static PTQ methods.
- Quantization errors in DLLMs accumulate geometrically across iterative denoising steps, with the attention mechanism's matrix multiplication identified as a primary source.
- Treating all tokens uniformly in weight quantization is suboptimal for DLLMs; distinguishing between stable unmasked tokens and probabilistic masked tokens is crucial for effective error compensation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.