Reproduce “TOMCAT: Test-time Comprehensive Knowledge Accumulation for Compositional Zero-Shot Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xudong Yan, Songhe Feng

TOMCAT introduces a framework for Compositional Zero-Shot Learning that addresses performance degradation from test-time label distribution shifts by continuously accumulating multimodal knowledge and adjusting prototypes from unlabeled test data. The approach achieved improved performance on four benchmark datasets in both closed-world and open-world settings, notably increasing AUC by up to 4.3% in the open-world scenario.

Key insights

  • Continuously adapting multimodal prototypes using unlabeled test data effectively mitigates label distribution shifts in Compositional Zero-Shot Learning.
  • Integrating both visual knowledge (from a dynamic priority queue of test images) and textual knowledge (from VLM embeddings) enhances the model's discriminative capabilities for novel compositions.
  • An adaptive weighting mechanism for prototype updates helps balance the retention of previously learned knowledge with the integration of new information, preventing catastrophic forgetting.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.