Reproduce “SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ioannis Kakogeorgiou, Spyros Gidaris, Konstantinos Karantzalos, Nikos Komodakis
An unsupervised object-centric learning framework, SPOT, enhances the decomposition of complex real-world images into distinct object entities. It achieves superior performance on challenging datasets by integrating a self-training mechanism with attention distillation and an autoregressive decoder leveraging patch-order permutations.
Key insights
- Autoregressive decoders, when making sequential predictions, often learn superior object segmentation boundaries compared to the encoder's direct slot-attention mechanism.
- Standard left-to-right autoregressive training can lead to decoders under-utilizing global slot information by excessively relying on previously predicted (or ground-truth) tokens.
- Self-training, by distilling the decoder's attention masks to the encoder, provides a stable mechanism to fine-tune powerful pre-trained image encoders for better object-centric representations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.