Reproduce “Unsupervised Part Discovery via Dual Representation Alignment” on OpenResearch

About this paper

Jiahao Xia, Wenjian Huang, Min Xu, Jianguo Zhang, Haimin Zhang, Ziyu Sheng, Dong Xu

This research introduces a dual representation alignment framework that enables Vision Transformers to learn part-specific attention in an unsupervised manner, leveraging image reconstruction as a pretext task. The method achieves competitive part discovery performance, often surpassing existing unsupervised and weakly-supervised approaches on multiple benchmarks by enhancing geometric invariance and semantic consistency.

Key insights

• Dual representation alignment effectively bridges sparse part-level representations from the PartFormer with dense pixel-level features, allowing learned part representations to function as adaptive detectors.
• Introducing specific geometric constraints, such as Concentration Loss, compels the PartFormer to focus locally and form coherent, part-specific attention maps.
• Exchanging part representations between geometrically transformed image pairs and enforcing semantic consistency significantly improves robustness to transformations and ensures consistent part identification.

Summary by alphaXiv.