Reproduce “KV Cache Quantization for Diffusion Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yan Tai, Yunan Ding, Guangzhao Li, Siqi Luo, Chaoyang Zhang, Guangtao Zhai, Guodong Guo, Xiaohong Liu
Fast-dLLM studies lower-bit KV cache quantization for diffusion language models, which reuse cached attention states as masked tokens are revealed. dLLM-KVQ calibrates for this reveal process and keeps selected key entries in high precision while quantizing the rest. On LLaDA-Instruct with 2-bit prefix-cache quantization, its four-task average is 44.38, versus 43.62 for KIVI and 41.45 for KVQuant, but below the unquantized average of 45.08.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.