Reproduce “Birth, Death and Diff: Insertion-Based Sequence Generation with Corrections” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Durga Prasad Maram, Dhruvesh Patel, Tahira Naseem, Benjamin Rozonoyer, Gaurav Pandey, Ramón Fernandez Astudillo, Andrew McCallum
δMDM lets insertion-based masked diffusion models revise sequences by substituting previously unmasked tokens and deleting newly inserted masks, alongside inserting and unmasking. Its proposal and correction passes support drafting with revision. On OpenWebText, at 1,024 network forward passes, it reaches generative perplexity 20.35, versus 40.37 for FlexMDM and 38.16 for MDM; it is slightly worse than FlexMDM at 32 passes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.