Reproduce “Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shu Zhang, Ran Xu, Caiming Xiong, Chetan Ramaiah
Salesforce Research developed a framework called "Use All The Labels" that integrates explicit hierarchical multi-label information directly into contrastive learning, leading to more generalizable and semantically meaningful embeddings. The approach, utilizing novel loss functions and a hierarchical batch sampling strategy, achieves superior performance in classification, retrieval, and clustering tasks, particularly on unseen data, while preserving hierarchical relationships in the embedding space.
Key insights
- Explicitly incorporating hierarchical multi-label information into contrastive learning yields more informative and nuanced embeddings.
- The framework ensures intrinsic preservation of hierarchical relationships within the learned embedding space, allowing embeddings to carry multi-level relational information.
- Learning representations that capture full hierarchical structure enhances generalizability to unseen data and improves performance across various downstream tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.