Reproduce “Revisiting Cross-Attention Mechanisms: Leveraging Beneficial Noise for Domain-Adaptive Learning” on OpenResearch

About this paper

Zelin Zang, Yehui Yang, Fei Wang, Liangyu Li, Baigui Sun

Researchers from Westlake University and Alibaba Group developed the Domain-Adaptive Cross-Scale Matching (DACSM) framework, which leverages beneficial noise in cross-attention for content-style disentanglement and a Cross-Scale Matching (CSM) module to handle scale discrepancies in Unsupervised Domain Adaptation. This framework achieved 91.7% accuracy on VisDA-2017, 81.3% on Office-Home, and 46.3% on DomainNet, outperforming previous state-of-the-art methods.

Key insights

• Cross-attention mechanisms can function as a domain translation tool, with queries primarily encoding semantic content and key-value pairs capturing style information.
• Injecting beneficial noise into the Key and Value matrices of cross-attention serves as a regularization technique, enhancing the model's focus on content features and improving cross-domain generalization.
• Jointly addressing both appearance and scale gaps through a combined framework yields a synergistic effect, resulting in more robust and discriminative domain-invariant feature learning.

Summary by alphaXiv.