Reproduce “Training-Free Model Merging for Multi-target Domain Adaptation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wenyi Li, Huan-ang Gao, Mingju Gao, Beiwen Tian, Rong Zhi, Hao Zhao
This paper introduces a training-free model merging technique for multi-target domain adaptation in semantic segmentation, overcoming the need for simultaneous access to all target domain data. The method leverages linear mode connectivity between domain-adapted models and proposes a principled approach to merge batch normalization statistics, achieving performance comparable to models trained with combined datasets.
Key insights
- Domain-adapted models derived from the same pretrained weights exhibit linear mode connectivity, allowing for simple averaging of parameters without significant performance loss.
- Batch normalization running statistics, which capture domain-specific characteristics, require a specialized merging approach based on Gaussian distribution assumptions for optimal performance.
- Sharing identical pretrained weights is crucial for establishing the common optimization landscape that enables successful model merging.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.