Reproduce “Inspecting class hierarchies in classification-based metric learning models” on OpenResearch

About this paper

Hyeongji Kim, Pekka Parviainen, Terje Berge, Ketil Malde

Researchers developed ProxyDR, a classification-based metric learning model, which demonstrates superior implicit learning of class hierarchies compared to NormFace and improved performance on hierarchy-aware metrics. The work also introduces a comprehensive set of evaluation measures and reveals an inherent hierarchical correspondence in nearly untrained CNNs.

Key insights

• ProxyDR, particularly with dynamic scaling, demonstrates superior ability to implicitly learn and structure class hierarchies compared to NormFace, even without explicit hierarchical training.
• Prototypes from nearly untrained CNNs (pretrained on ImageNet, one epoch) exhibit a measurable correspondence with predefined semantic hierarchies, suggesting an inherent inductive bias.
• There is a observed trade-off between maximizing hierarchy-informed performance (especially with explicit hierarchy training like MDS or CORR loss) and maintaining high standard top-5 classification accuracy.

Summary by alphaXiv.