Reproduce “Domain-Specificity Inducing Transformers for Source-Free Domain Adaptation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sunandini Sanyal, Ashish Ramayee Asokan, Suvaansh Bhambri, Akshay Kulkarni, Jogendra Nath Kundu, R. Venkatesh Babu
Researchers from the Indian Institute of Science, Bengaluru, developed the Domain-Specificity inducing Transformer (DSiT) framework for Source-Free Domain Adaptation, which explicitly disentangles domain-specific and task-specific factors within a Vision Transformer. This approach achieved state-of-the-art performance across single-source, multi-source, and multi-target SFDA benchmarks, including a 5.1% improvement on DomainNet over the non-source-free CDTrans.
Key insights
- In-domain specificity, rather than just domain invariance, is crucial for achieving superior performance in domain adaptation settings.
- Disentangling domain-specific factors from task-specific factors allows for better control over learning and leveraging both types of information without mutual degradation.
- Vision Transformer query weights can be effectively specialized to capture domain-specific information, offering a novel mechanism for disentanglement within transformer architectures.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.