Reproduce “AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Size” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Guanxi Lu, Hao Mark Chen, Yuto Karashima, Zhican Wang, Daichi Fujiki, Hongxiang Fan
Researchers from Imperial College London and the Institute of Science Tokyo developed AdaBlock-dLLM, a training-free scheduler that dynamically adjusts block sizes for semi-autoregressive diffusion LLM inference based on semantic coherence. This approach improves generation accuracy by up to 5.3% on benchmarks like GSM8K while maintaining comparable throughput, especially when combined with KV caching.
Key insights
- Denoising in dLLMs reveals distinct confidence dynamics, including local high-confidence plateaus and a critical 'Volatility Band' where confidence is unstable.
- The variability within the Volatility Band indicates that fixed block sizes are misaligned with the model's inductive decoding preferences and local stochasticity.
- Dynamically adjusting block sizes based on semantic steps, marked by high-confidence delimiter tokens, significantly mitigates late decoding overhead and premature decoding errors.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.