Reproduce “Generalized Parametric Contrastive Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jiequan Cui, Zhisheng Zhong, Zhuotao Tian, Shu Liu, Bei Yu, Jiaya Jia

Researchers from The Chinese University of Hong Kong and SmartMore introduced Generalized Parametric Contrastive Learning (GPaCo), a framework that rebalances supervised contrastive loss using learnable class centers, enabling state-of-the-art performance on long-tailed datasets while also boosting accuracy on balanced datasets and improving out-of-distribution robustness.

Key insights

  • Theoretical analysis demonstrates that conventional SupCon loss inherently biases towards high-frequency classes, resulting in a higher lower bound of loss for these classes.
  • PaCo's modified loss function adaptively rebalances learning signals, allowing the model to pay more attention to low-frequency classes and effectively learn hard examples in balanced settings.
  • The momentum encoder, a common component in contrastive learning, is not essential and can even be detrimental in supervised contrastive learning, with its removal leading to the improved GPaCo.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.