Reproduce “Boosting Unsupervised Domain Adaptation with Soft Pseudo-label and Curriculum Learning” on OpenResearch

About this paper

Shengjia Zhang, Tiancheng Lin, Yi Xu

Researchers from Shanghai Jiao Tong University developed SPCL, a two-stage learning framework that enhances existing Unsupervised Domain Adaptation (UDA) algorithms by utilizing soft pseudo-labels and a curriculum learning strategy. This framework consistently improved classification accuracy on benchmark datasets, boosting performance for various UDA methods, such as an average accuracy gain of 2.4% for CAN on Office-Home.

Key insights

• Using soft pseudo-labels, rather than hard one-hot labels, provides richer semantic information and acts as label smoothing, effectively reducing the pseudo-label inaccuracy ratio and improving target domain supervision.
• An adaptive curriculum learning strategy, which gradually increases the weight of the target domain loss, effectively prevents source domain overfitting and ensures the model focuses on adapting to the target distribution as pseudo-labels become more reliable.
• Initializing the second-stage network with ImageNet pre-trained weights is crucial for better generalization and avoiding local optima that the initial adapted model might be trapped in.

Summary by alphaXiv.