Reproduce “SlotDiT: Object-Centric Representations for Diffusion Transformers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Gjergj Plepi, Sven Behnke
SlotDiT replaces dense VAE-style video latents in diffusion transformers with compact object-centric slots for language-conditioned robotic prediction. Across four manipulation datasets, it generally produces lower perceptual video scores than VideoVAE but improves task-oriented outcomes, reaching up to 87.8% video task success, 74.5% closed-loop control success, and 5.68× faster generation in the reported CLIPort comparison.
Key insights
- Task completion and perceptual quality can diverge: VideoVAE often achieves better LPIPS and FVD, whereas SlotDiT more reliably preserves the object-level structure needed for instruction following and manipulation.
- Using multiple slots is essential; replacing them with a single latent reduces task success from 57.6% to 33.0% on BridgeData V2 and from 79.7% to 12.7% on LanguageTable-Synthetic.
- The slot representation uses 10 tokens per frame instead of 256 for the main baselines, enabling reported generation at 9.33 frames per second versus 1.64 frames per second for DiT with SD-VAE.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.