Reproduce “CoNMix for Source-free Single and Multi-target Domain Adaptation” on OpenResearch

About this paper

Vikash Kumar, Rohit Lal, Himanshu Patil, Anirban Chakraborty

CoNMix, developed by researchers at the Indian Institute of Science, introduces a three-stage framework for source-free single and multi-target domain adaptation, leveraging Vision Transformers and MixUp knowledge distillation. The method establishes new state-of-the-art performance for source-free single-target domain adaptation and provides the first comprehensive baselines for the novel source-free multi-target domain adaptation task, demonstrating robust generalization across various datasets.

Key insights

• Vision Transformers (VTs) are empirically shown to provide superior feature representations for domain adaptation compared to CNNs, exhibiting better domain transferability and class-discriminative features.
• Nuclear-Norm Maximization is a critical component for effective source-free adaptation, enhancing both class-discriminability and prediction diversity on unlabeled target data, especially when combined with pseudo-label refinement.
• MixUp Knowledge Distillation effectively enables a single student model to generalize across multiple diverse target domains in a source-free setting by leveraging mixed inputs and pseudo-labels from domain-specific teacher models.

Summary by alphaXiv.