Reproduce “Vision Transformers for Zero-Shot Clustering of Animal Images: A Comparative Benchmarking Study” on OpenResearch

About this paper

Hugo Markoff, Stefan Hein Bengtson, Michael Ørsted

A framework combining Vision Transformer models with dimensionality reduction and clustering algorithms successfully organizes unlabeled camera trap images into species-level groups. The approach achieved a V-measure of 0.943 for unsupervised clustering, substantially reducing the manual annotation burden in biodiversity monitoring.

Key insights

• Self-supervised DINO models (DINOv3 and DINOv2) significantly outperformed biology-specific (BioCLIP 2) and general-purpose vision-language models for generating robust, transferable embeddings suitable for clustering.
• Dimensionality reduction was crucial for optimal clustering performance, with t-SNE consistently improving V-measure by 26–38 percentage points compared to direct clustering on raw high-dimensional embeddings.
• HDBSCAN emerged as the superior unsupervised clustering algorithm, demonstrating competitive performance, accurate cluster count prediction, and a low outlier rate, especially when combined with DINOv3 and t-SNE.

Summary by alphaXiv.