Reproduce “RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression” on OpenResearch

About this paper

Yu Gong, Greg Mori, Frederick Tung

RankSim introduces a regularization technique that explicitly aligns the ranking of neighbors in label space with their learned feature space representations, addressing deep imbalanced regression. The method achieves new state-of-the-art performance on age estimation and sentence similarity benchmarks, notably improving generalization on few-shot and zero-shot data and reducing Mean Absolute Error to 6.91 on the AgeDB-DIR dataset.

Key insights

• Aligning the global ranking of neighbors in the learned feature space with the label space provides a stronger inductive bias for deep imbalanced regression than local distribution smoothing.
• The inherent ordering and continuity of regression labels can be more comprehensively leveraged by regularizing the full spectrum of label-space relationships, from near to distant.
• Leveraging methods for backpropagation through blackbox combinatorial solvers allows for the effective integration of non-differentiable ranking operations into deep learning training.

Summary by alphaXiv.