Reproduce “Improving Discrete Diffusion Unmasking Policies Beyond Explicit Reference Policies” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Chunsan Hong, Seonho An, Min-Soo Kim, Jong Chul Ye

Researchers at KAIST developed a method to learn adaptive unmasking policies for frozen discrete diffusion models (MDMs) by formulating the denoising process as a KL-regularized Markov Decision Process. This approach consistently outperformed established heuristic unmasking strategies across various logic puzzles and mathematical reasoning benchmarks, improving accuracy by 11.2% over max-confidence on SUDOKU and 1.9% on GSM8K.

Key insights

  • Learning an adaptive unmasking policy can consistently outperform fixed heuristic strategies by discovering more effective unmasking sequences across diverse tasks.
  • Formulating the denoising process as a KL-regularized MDP provides theoretical guarantees for policy improvement and convergence to samples statistically closer to the true data distribution.
  • Incorporating a strong reference policy and a KL-divergence regularization term stabilizes policy optimization, leading to better exploration and higher final accuracy in complex search spaces.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.