Reproduce “Level-of-Token Diffusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kiyohiro Nakayama, Brian Chao, Jan Ackermann, Hansheng Chen, Federico Tombari, Leonidas Guibas, Lior Yariv, Gordon Wetzstein
LoT allocates image and video transformer tokens according to desired detail, using larger patches in simple regions and smaller patches where detail matters. A transformer processes the shorter sequence, while extent-aware flow recovery produces a full-resolution denoising field. At matched 2× video token budget, LoT reports 2.262× speedup and higher Aesthetic and Imaging scores than Foveated Diffusion, but not higher Dynamic Degree at the highest tested compression.
Key insights
- Token allocation can serve as both a compute-saving mechanism and a way to steer where generated detail appears.
- The approach preserves a full-resolution latent output even though the transformer processes fewer tokens; its efficiency depends on layouts that correctly identify regions able to tolerate coarser processing.
- Ablations show that patch-size embeddings and analytic asymmetric-flow recovery support performance, particularly under more aggressive compression.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.