Reproduce “Accelerating Diffusion LLM Inference via Local Determinism Propagation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Fanheng Kong, Jingyuan Zhang, Yahui Liu, Zirui Wu, Yu Tian, Victoria W., Guorui Zhou
Researchers from Kuaishou Technology introduced LocalLeap, a training-free adaptive parallel decoding strategy for diffusion large language models (dLLMs) that significantly boosts inference speed. It achieves up to 6.94x throughput improvement on benchmarks like GSM8K while maintaining or even slightly improving generation quality.
Key insights
- A significant portion of tokens in dLLMs reach stable predictions early in the decoding process, a phenomenon termed 'delayed decoding', indicating wasted computation.
- High-confidence tokens ('anchors') locally propagate determinism, stabilizing predictions of nearby masked tokens and enabling their early, parallel commitment.
- The reliability of predictions decreases with distance from a high-confidence anchor, necessitating a bounded spatial radius for relaxed parallel decoding.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.