Reproduce “Image Clustering with External Guidance” on OpenResearch

About this paper

Yunfan Li, Peng Hu, Dezhong Peng, Jiancheng Lv, Jianping Fan, Xi Peng

Researchers introduce the "externally guided clustering" paradigm through Text-Aided Clustering (TAC), leveraging textual semantics from WordNet and Vision-Language Pre-trained models like CLIP to enhance image clustering. This method achieves state-of-the-art performance across multiple benchmarks, often outperforming zero-shot CLIP by utilizing a cross-modal mutual distillation strategy for improved feature discriminability and achieving up to 43.5% ARI gains on datasets like ImageNet-Dogs.

Key insights

• External knowledge, particularly textual semantics, can provide more discriminative and powerful supervision signals for image clustering than relying solely on internal data-driven approaches.
• Pre-trained Vision-Language models (VLPs) like CLIP can be effectively repurposed for unsupervised clustering by constructing semantic bridges (text counterparts) and facilitating cross-modal knowledge transfer, extending beyond their conventional zero-shot classification role.
• Even a simple concatenation of image features with externally derived text counterparts (TAC without training) provides substantial performance gains, indicating the inherent value of external semantic information.

Summary by alphaXiv.