Reproduce “Counterfactual Explanations for Continuous Action Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shuyang Dong, Shangtong Zhang, Lu Feng

A method is presented for generating counterfactual explanations for Reinforcement Learning (RL) agents in continuous action spaces. It extends the TD3 algorithm to identify alternative action sequences that yield superior outcomes with minimal deviation from observed actions, and accommodates domain-specific constraints.

Key insights

  • Generating continuous action counterfactuals can be formulated as an RL optimization problem by incorporating action distance penalties into the reward function.
  • Training in diverse multi-environment settings substantially improves the generalization capability of counterfactual explanations, leading to the discovery of more advantageous outcomes.
  • The performance of different constraint-handling policies is sensitive to the environment's dynamics, specifically how immediately or delayed action effects manifest.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.