Reproduce “Hierarchical Continuous Diffusion Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hui Ren, Zihan Li, Chang Liu, Huidong Liu, Alexander Schwing
HC-DLM couples a persistent continuous latent with token readouts that are re-noised and fed back to guide each next latent update. The authors derive a variational lower bound and train with reconstruction, encoder-entropy, and conditional denoising losses. On Hard Sudoku, the 6M-parameter model reaches 72.41% exact-solution accuracy, versus 24.74% without discrete conditioning, in this tested setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.