Reproduce “Normative Alignment of Recommender Systems via Internal Label Shift” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Johannes Kruse, Kasper Lindskow, Michael Riis Andersen, Ryotaro Shimizu, Julian McAuley, Pierre-Alexandre Mattei, Jes Frellsen
NAILS reweights a personalized recommender’s item probabilities by category to steer the aggregate recommendation mix toward a platform target, without retraining the model. In an offline EB-NeRD news experiment, deterministic NAILS at λ=.7 reduced aggregate top-10 KL for the editorial target to .0540, versus .2766 for the baseline. This finding is limited to one target and aggregate recommendations across users.
Key insights
- Normative alignment can be achieved by framing it as an internal label shift, allowing for a principled and efficient reweighting of item probabilities to match desired attribute distributions.
- Aligning recommender outputs with certain normative targets, such as editorially curated distributions, can be synergistic with user engagement, potentially improving traditional ranking metrics (AUC).
- There exists an inherent trade-off between maximizing traditional ranking performance and achieving specific normative goals, particularly with objectives like uniform content distribution, highlighting the importance of careful target distribution design.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.