Reproduce “Virtual Class Enhanced Discriminative Embedding Learning” on OpenResearch

About this paper

Binghui Chen, Weihong Deng, Haifeng Shen

Researchers from Beijing University of Posts and Telecommunications and Didi Chuxing's AI Labs propose Virtual Softmax, an elegant modification to the standard Softmax loss that automatically enforces more discriminative feature learning. This method integrates a dynamically constructed virtual negative class into the loss calculation, yielding improved feature separation and compactness, demonstrated by gains such as a 0.28% error rate on MNIST and a 99.46% accuracy on LFW for face verification.

Key insights

• Placing a single, dynamically constructed virtual negative class into the Softmax denominator forces original class boundaries to separate, leading to larger angular margins and tighter intra-class clusters.
• The virtual class anchor is constructed to align with the current feature vector and match the magnitude of the true class anchor, creating a strong 'zero-limit' constraint that forces perfect alignment between features and their class anchors.
• Virtual Softmax provides implicit regularization, acting as an adaptive negative sample that eliminates the need for manual margin tuning or complex training procedures found in prior margin-based methods.

Summary by alphaXiv.