Reproduce “Equipping Vision Foundation Model with Mixture of Experts for Out-of-Distribution Detection” on OpenResearch

About this paper

Shizhen Zhao, Jiahui Liu, Xin Wen, Haoru Tan, Xiaojuan Qi

Researchers from The University of Hong Kong propose the Mixture of Feature Experts (MoFE) module and Dynamic-β Mixup data augmentation to equip vision foundation models, particularly DINOv2, for enhanced out-of-distribution detection. This approach achieved state-of-the-art performance with 17.01% FPR95 and 95.89% AUROC on average across ImageNet-1K OOD benchmarks.

Key insights

• DINOv2, even without fine-tuning, inherently provides a highly discriminative feature space for zero-shot OOD detection, outperforming other VFMs and some fine-tuned methods on challenging fine-grained OOD datasets.
• Decoupling complex in-domain feature distributions into specialized subspaces via model-centric experts (MoFE) simplifies optimization, leading to more compact ID distributions and clearer OOD decision boundaries.
• Dynamically adapting data augmentation (Dynamic-β Mixup) based on category discriminativeness is crucial for VFMs, preventing over-smoothing for well-represented classes while facilitating learning for less discriminative ones.

Summary by alphaXiv.