Reproduce “Learning to Predict Visual Attributes in the Wild” on OpenResearch

About this paper

Khoi Pham, Kushal Kafle, Zhe Lin, Zhihong Ding, Scott Cohen, Quan Tran, Abhinav Shrivastava

The paper introduces Visual Attributes in the Wild (VAW), a large-scale dataset featuring dense positive and negative annotations with segmentation masks for general visual attribute prediction. It also proposes SCoNE, an algorithm incorporating object-aware attention, linguistic-based negative label expansion, and multi-label supervised contrastive learning, which improves attribute prediction performance on VAW, particularly for long-tail and low-level attributes.

Key insights

• Explicit negative labels are crucial for training robust attribute classifiers, providing significant performance gains on large-scale, partially annotated datasets.
• Combining precise object localization, guided by segmentation masks, with multi-attention mechanisms allows models to effectively focus on relevant object regions and contextual cues for diverse attribute types.
• Adapting supervised contrastive learning to a multi-label setting, by learning attribute-aware embeddings and contrasting them, enhances feature discriminability, which is particularly beneficial for imbalanced data distributions.

Summary by alphaXiv.