Reproduce “Adversarial Counterfactual Environment Model Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xiong-Hui Chen, Yang Yu, Zheng-Mao Zhu, Zhihua Yu, Zhenjun Chen, Chenghe Wang, Yinan Wu, Hongqiu Wu and 3 more

GALILEO, an adversarial approach for learning environment dynamics models, robustly predicts outcomes for counterfactual actions despite selection bias in offline data by minimizing errors against adversarially constructed distributions. This method improves prediction accuracy and significantly enhances performance in off-policy evaluation, offline policy improvement, and real-world online decision-making applications.

Key insights

  • Explicitly minimizing model error against adversarially chosen, error-maximizing data distributions forces the model to generalize robustly beyond the observed offline data.
  • The optimal adversarial distribution concentrates on state-action pairs where the current model deviates most significantly from the true dynamics or where the true environment is inherently more stochastic.
  • A GAN-like architecture provides a tractable solution to the AWRM objective by training discriminators to estimate density ratios for weighting data, making the approach applicable to high-dimensional problems.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.