Reproduce “Model Reuse with Reduced Kernel Mean Embedding Specification” on OpenResearch

About this paper

Xi-Zhu Wu, Wenkai Xu, Song Liu, Zhi-Hua Zhou

The authors propose reduced kernel mean embeddings (RKMEs): compact, constructed summaries of models’ input distributions that let users match trained models to unlabeled data without sharing raw examples. A closest model handles recurring tasks, while a selector combines models for mixtures. The method reaches 86.22% versus HMR’s 70.58% on recurring CIFAR-100 tasks, but only under solved-distribution assumptions.

Key insights

• Reduced Kernel Mean Embedding (RKME) can serve as an effective and privacy-preserving specification for pre-trained models by representing their underlying data distributions without exposing raw training data.
• The framework provides theoretical guarantees and practical mechanisms for reusing models under both task-recurrent (new task matches an existing one) and instance-recurrent (new task is a mixture of existing ones) assumptions.
• By leveraging kernel herding, the instance-recurrent deployment phase can generate a labeled 'mimic sample set' that reflects the target task's mixed distribution, enabling the training of an adaptive model selector without source data access.

Summary by alphaXiv.