Reproduce “Keep It Light! Simplifying Image Clustering Via Text-Free Adapters” on OpenResearch

About this paper

Yicen Li, Haitz Sáez de Ocáriz Borde, Anastasis Kratsios, Paul D. McNicholas

Researchers at McMaster University introduced Simple Clustering via Pre-trained models (SCP), a text-free deep clustering method that achieves state-of-the-art performance by using a frozen pre-trained vision backbone and a lightweight trainable cluster head. SCP-MIX achieved 96.5% accuracy on CIFAR-10 and 99.3% on ImageNet-10, demonstrating that competitive image clustering can be performed without complex multi-modal architectures or textual data.

Key insights

• The Lossless Amortization Principle (LAR) and Theorem 1 theoretically justify that explicit textual information might not be fundamentally necessary for strong clustering performance, as image semantics can be sufficiently captured by visual features alone.
• Leveraging pre-trained visual backbones (e.g., CLIP, DINO) and a simple training objective focused on positive pair consistency with regularization can achieve high clustering accuracy without complex architectures or negative pairs.
• The entropy regularization term is critical for preventing model collapse, and the confidence loss contributes to improved performance, especially in scenarios with a larger number of clusters.

Summary by alphaXiv.