Reproduce “Counterfactual harm” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jonathan G. Richens, Rory Beard, Daniel H. Thompson
Counterfactual Harm formalizes harm as the utility loss an individual would have avoided under a specified default action, rather than treating every adverse outcome as intervention-caused. The framework shows that factual objectives such as expected utility, treatment effects, and risk-sensitive rewards can select harmful policies after distribution shifts, while a counterfactual harm penalty supports safer choices; in an Aripiprazole dose model, it reduced estimated harm by about 90% for only a roughly 2% reduction in treatment effect.
Key insights
- Two treatments can have identical recovery rates or treatment effects while differing substantially in harm: one may help only those who would otherwise remain ill, whereas another may kill people who would have recovered without intervention.
- No objective based solely on factual action-outcome distributions can guarantee harm avoidance under all distribution shifts, because environments with identical factual statistics can have different factual-counterfactual relationships.
- Risk aversion and harm aversion address different problems: reducing outcome variance does not necessarily reduce the adverse outcomes causally attributable to an agent.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.