Reproduce “Probabilistic Contrastive Learning for Long-Tailed Visual Recognition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chaoqun Du, Yulin Wang, Shiji Song, Gao Huang
Probabilistic Contrastive Learning (ProCo) models feature distributions for each class using von Mises-Fisher distributions to derive a closed-form expected contrastive loss. This approach addresses the large batch size dependency of traditional supervised contrastive learning, leading to improved accuracy across long-tailed image classification, semi-supervised learning, and object detection tasks.
Key insights
- Modeling features with vMF distributions on the unit hypersphere accurately captures class-specific feature distributions suitable for contrastive learning.
- Deriving an *expected* contrastive loss in closed form allows for efficient training without the need for large batch sizes, mitigating a key limitation of existing SCL methods.
- The inherent structure of the ProCo loss naturally incorporates a form of margin modification, contributing to better performance on minority classes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.