Reproduce “D5P4: Partition Determinantal Point Process for Diversity in Parallel Discrete Diffusion Decoding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jonathan Lys, Vincent Gripon, Axel Marmoret, Lukas Mauch, Fabien Cardinaux, Ghouthi Boukli Hacene, Bastien Pasdeloup
Researchers at IMT Atlantique and Sony Europe developed D5P4, a beam-style decoding method for discrete diffusion models that leverages Partition Determinantal Point Processes to enhance output diversity. D5P4 achieved superior quality-diversity trade-offs, significantly boosted set coverage in question answering and mathematical reasoning tasks, and improved computational efficiency by reducing categorical sampling time by nearly 10-fold and peak GPU memory by 40%.
Key insights
- Determinantal Point Processes can be effectively adapted to the parallel nature of discrete diffusion, enabling principled control over the balance between generation quality and output diversity.
- Reliable internal signals from diffusion models (normalized negative entropy for quality and flattened hidden representations for diversity) are sufficient to guide a diverse decoding process without needing external evaluators.
- Explicitly enforcing a transversal partition constraint during candidate selection is critical for mitigating lineage collapse, thereby ensuring consistent and reliable diversity across iterative denoising steps.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.