Reproduce “Recommendations as Treatments: Debiasing Learning and Evaluation” on OpenResearch

About this paper

Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, Thorsten Joachims

Schnabel et al. from Cornell University introduce a causal inference framework to address selection bias in recommender systems, enabling unbiased evaluation and learning. Their Inverse-Propensity-Scoring (IPS) and Self-Normalized Inverse Propensity Scoring (SNIPS) estimators yield orders of magnitude more accurate performance assessments, while the propensity-weighted matrix factorization (MF-IPS) method demonstrates substantial improvements over unweighted models and existing imputation techniques on real-world datasets.

Key insights

• Applying causal inference principles, specifically propensity scoring, offers a robust and theoretically sound method to debias both evaluation and learning in recommender systems.
• IPS and SNIPS estimators can provide orders of magnitude more accurate performance assessments compared to naive estimators, even when dealing with highly biased data.
• Propensity weighting can be effectively integrated into standard machine learning objectives like matrix factorization, leading to demonstrably better and more generalized predictive models.

Summary by alphaXiv.