Reproduce “PBPO: Prefix-Block Policy Optimization for Diffusion Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Katherine Tieu, Dongqi Fu, Si Zhang, Hanqing Zeng, Tianxin Wei, Chenglin Wei, Yinglong Xia, Hong Li and 1 more
PBPO trains masked diffusion language models with block-wise reinforcement learning. It estimates token probabilities at each block boundary, then clips each token’s importance weight while retaining shared block context. In a controlled MATH500 comparison, PBPO remains more stable than block-level weighting, whose reward collapses; this result covers two LLaDA-family initializations and relies on a one-step surrogate, not exact denoising likelihoods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.