Reproduce “Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests” on OpenResearch

About this paper

Victor Veitch, Alexander D'Amour, Steve Yadlowsky, Jacob Eisenstein

This research introduces a causal inference framework to formalize "stress tests" as counterfactual invariance, a property where a model's prediction remains unchanged when irrelevant input aspects are perturbed. The work develops regularization methods that achieve this invariance, leading to improved out-of-domain performance by aligning the regularization strategy with the data's causal structure.

Key insights

• The appropriate method for mitigating spurious correlations is fundamentally determined by the causal structure of the data, with distinct structures (e.g., anti-causal vs. causal direction) requiring different observable independence conditions.
• Counterfactual invariance can be approximated and enforced through observable conditional independence constraints, offering a practical pathway to robust models without direct counterfactual observations.
• Achieving counterfactual invariance provides theoretical guarantees for model robustness against specific types of domain shifts where only spurious associations, and not fundamental causal mechanisms, change.

Summary by alphaXiv.