Reproduce “LaViDa: A Large Diffusion Language Model for Multimodal Understanding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shufan Li, Konstantinos Kallidromitis, Hritik Bansal, Akash Gokul, Yusuke Kato, Kazuki Kozuka, Jason Kuen, Zhe Lin and 2 more
LaViDa connects an image encoder to a masked diffusion language model that generates responses by iteratively revealing tokens. Complementary masking trains on disjoint masked spans, while Prefix-DLM caches image-and-question representations and timestep shifting controls reveal schedules. It reports 1.00 sentence- and sample-level constraint-satisfaction rates on 100 poem samples, but OCR and some benchmark results lag stronger models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.