Reproduce “Learning the Difference that Makes a Difference with Counterfactually-Augmented Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Divyansh Kaushik, Eduard Hovy, Zachary C. Lipton
Researchers at Carnegie Mellon University developed a human-in-the-loop counterfactual data augmentation method that effectively reduces machine learning models' reliance on spurious correlations. This approach, applied to sentiment analysis and natural language inference tasks, leads to improved robustness and out-of-domain generalization, with models exhibiting up to a 36-point increase in accuracy on counterfactual test sets compared to models trained without augmentation.
Key insights
- Spurious patterns can be coherently defined within a causal framework, representing associations due to confounding rather than direct or indirect causal effects.
- Human-in-the-loop counterfactual revision, guided by a "least action principle" (making only necessary changes), can effectively disentangle causally relevant features from spurious correlations in text.
- Training models on a combination of original and counterfactually-augmented data forces them to learn more robust and generalizable features by breaking up spurious correlations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.