Reproduce “Calibration-compatible Listwise Distillation of Privileged Features for CTR Prediction” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xiaoqiang Gui, Yueyao Cheng, Xiang-Rong Sheng, Yunfeng Zhao, Guoxian Yu, Shuguang Han, Yuning Jiang, Jian Xu and 1 more

Researchers from Shandong University and Alibaba Group introduced Calibration-compatible LIstwise Distillation (CLID), a method that leverages privileged features for Click-Through Rate (CTR) prediction by enabling listwise knowledge distillation while preserving model calibration. Experiments on public and Alibaba production datasets demonstrated CLID improved ranking performance (e.g., +0.38% GAUC on production data) without compromising calibration.

Key insights

  • A formal definition of "calibration-compatible property" for distillation loss was introduced, ensuring a student model can optimize both for knowledge transfer and true probability estimation.
  • The proposed CLID's specific listwise loss formulation was theoretically proven to achieve its global minima simultaneously with pointwise cross-entropy losses of both student and teacher, thereby preserving calibration.
  • Traditional listwise losses were shown to improve ranking but significantly harm calibration when directly applied as distillation losses in CTR prediction, highlighting a key dilemma CLID addresses.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.