Reproduce “Unified Language-driven Zero-shot Domain Adaptation” on OpenResearch

About this paper

Senqiao Yang, Zhuotao Tian, Li Jiang, Jiaya Jia

This paper introduces Unified Language-driven Zero-shot Domain Adaptation (ULDA), a framework that enables a single model to adapt to multiple unseen target domains without needing domain-ID knowledge during inference. The approach relies solely on source images and textual descriptions of target domains during training, outperforming existing zero-shot methods like PØDA in unified adaptation scenarios and demonstrating robustness across various domain shifts, including challenging rare conditions.

Key insights

• Hierarchical Context Alignment (HCA) enables multi-level semantic alignment, ranging from scene-level to pixel-wise, between visual features and text descriptions across domains.
• Domain Consistent Representation Learning (DCRL) preserves the relational consistency between different categories across diverse target domains by aligning covariance matrices of prototypes and text embeddings.
• The Text-Driven Rectifier (TDR) refines simulated target domain features using parameters derived directly from target domain text embeddings, bridging the gap between simulated and real feature distributions.

Summary by alphaXiv.