Reproduce “RankingSHAP -- Listwise Feature Attribution Explanations for Ranking Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Maria Heuss, Maarten de Rijke, Avishek Anand

Researchers from the University of Amsterdam and Delft Institute of Technology developed RankingSHAP, a method that extends the SHAP framework to generate listwise feature attributions for ranking models. This approach enables the explanation of an entire ranked list's order or specific document exposures, evaluated using two distinct faithfulness paradigms.

Key insights

  • Listwise masking and flexible explanation objectives enable a more accurate understanding of how features influence the relative ordering of documents.
  • Pointwise feature attributions are inadequate for ranking tasks, as features influencing individual document scores may not reflect their impact on the overall listwise order or relative placements.
  • The use of specific listwise explanation objectives (e.g., Kendall's tau, weighted rank difference, exposure) allows for targeted insights into different aspects of a ranking decision, from overall order to individual document visibility.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.