Reproduce “Uncovering Bias in Foundation Models: Impact, Testing, Harm, and Mitigation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shuzhou Sun, Li Liu, Yongxiang Liu, Zhen Liu, Shuanghui Zhang, Janne Heikkilä, Xiang Li
This paper presents TriProTesting, a novel methodology for comprehensively detecting explicit, implicit, and intersectional biases in foundation models. It also introduces AdaLogAdjustment, an efficient post-processing technique for bias mitigation. The research empirically demonstrates pervasive biases in models like CLIP and ALIGN, and shows that AdaLogAdjustment successfully mitigates biases in over 99% of tested scenarios without requiring model retraining.
Key insights
- Biases are ubiquitous and often amplified when multiple social attributes intersect in foundation models.
- Models exhibit both explicit biases, detectable with positive and negative probes, and significant implicit biases, revealed through neutral probes.
- Bias mitigation can be achieved through computationally efficient post-processing adjustments that redistribute model "probability power," aligning with social science concepts of counter-labeling.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.