Reproduce “Unbiased Offline Evaluation of Contextual-bandit-based News Article Recommendation Algorithms” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Lihong Li, Wei Chu, John Langford, Xuanhui Wang

The authors introduce replay to evaluate news recommendation policies from randomized logs. It presents each logged context to a candidate and retains each interaction only when the candidate selects the logged article, using its observed click or non-click. In Yahoo! tests, replay estimates closely tracked online bucket measurements, including cross-policy comparisons, although the formal guarantee assumes i.i.d. events and fixed arms.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.