Reproduce “Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yangyang Zhong, Yanmei Gu, Zhengqing Zang, Xiaomeng Li, Yuqi Ding, Xibei Jia, Yuting Shen, Zhenzhong Lan and 9 more

The authors analyze masked diffusion language models, which can finalize multiple tokens per step, and compare them with autoregressive models across 58 benchmarks. They introduce Average Finalization Parallelism and Kendall’s tau to measure parallelism and generation order separately. Diffusion models trail autoregressive comparators overall, but a 100B model solves 78 of 100 Sudoku puzzles versus 63 for an 80B model, limited to this test.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.