Reproduce “Counterfactual Risk Minimization: Learning from Logged Bandit Feedback” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Adith Swaminathan, Thorsten Joachims

The paper presents POEM, a counterfactual-learning method that uses logged choice probabilities to estimate how new stochastic policies would perform. It minimizes clipped propensity-weighted loss with a penalty for estimator variability, producing a conservative risk objective. On simulated multi-label logs, POEM(B) achieves TMC Hamming loss 2.197 versus 2.808 for IPS(B), but requires exploration and does not match the full-information CRF.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.