Reproduce “Off-policy Policy Evaluation For Sequential Decisions Under Unobserved Confounding” on OpenResearch

About this paper

Hongseok Namkoong, Ramtin Keramati, Steve Yadlowsky, Emma Brunskill

Researchers from Stanford University developed a framework for Off-Policy Policy Evaluation (OPE) in sequential decision-making that accounts for unobserved confounding, demonstrating that a "one-decision confounding" model yields tractable, robust worst-case performance bounds. Their method quantifies the impact of unobserved factors on policy estimates and provides more informative robustness certificates than previous naive approaches in simulated healthcare settings.

Key insights

• Even small amounts of unobserved confounding affecting every decision in a sequence lead to an exponential increase in Off-Policy Policy Evaluation (OPE) bias with the horizon length.
• A "one-decision confounding" model, where unobserved factors influence only a single decision, provides a more realistic and analytically tractable framework for robust sequential OPE.
• Functional convex duality can be effectively utilized to transform the complex, infinite-dimensional problem of deriving worst-case bounds under confounding into a computationally feasible, finite-dimensional loss minimization problem.

Summary by alphaXiv.