Reproduce “AEFS: Adaptive Early Feature Selection for Deep Recommender Systems” on OpenResearch

About this paper

Fan Hu, Gaofeng Lu, Jun Chen, Chaonan Guo, Yuekui Yang, Xirong Li

Researchers from Renmin University of China and Tencent developed Adaptive Early Feature Selection (AEFS), a method for Deep Recommender Systems that dynamically selects relevant features before the embedding layer. AEFS maintains predictive accuracy comparable to existing adaptive approaches while reducing activated embedding parameters by 37.5%, leading to faster training and inference.

Key insights

• It is possible to combine the adaptive nature of instance-wise feature selection with the parameter efficiency of early-stage selection in deep recommender systems.
• A lightweight auxiliary model, with significantly smaller embedding dimensions, can effectively guide a larger main model's feature selection when coordinated through specific alignment losses.
• Both fine-grained embedding alignment loss ($L_{EA}$) and coarse-grained prediction alignment loss ($L_{PA}$) are critical for ensuring the auxiliary model provides relevant guidance without compromising the main model's performance.

Summary by alphaXiv.