Reproduce “Position Bias Estimation for Unbiased Learning-to-Rank in eCommerce Search” on OpenResearch

About this paper

Grigor Aslanyan, Utkarsh Porwal

Researchers at eBay Inc. developed a non-invasive method to estimate click propensities for unbiased Learning-to-Rank in e-commerce search, leveraging natural rank variations of query-document pairs over time. Their approach accurately estimated propensities up to rank 500 and improved the Area Under the Curve (AUC) of unbiased LambdaMART models by up to 1.2% over existing bias correction methods.

Key insights

• Click propensities can be accurately estimated without live search interventions or reliance on explicit relevance modeling by leveraging naturally occurring rank changes over time.
• The derived simplified likelihood function, achieved by conditioning on at least one click and assuming small click probabilities, effectively removes nuisance relevance parameters, making the estimation robust and efficient.
• Click propensities for mobile search exhibit distinct periodic patterns related to interface loading in batches (e.g., drops around rank 25, rise around rank 40), suggesting unique platform-specific user scrolling behaviors.
• The method enables reliable propensity estimation for significantly deeper ranks (up to 500) than previously possible, addressing a critical need for e-commerce platforms with extensive product catalogs.

Summary by alphaXiv.