Reproduce “FairerCLIP: Debiasing CLIP's Zero-Shot Predictions using Functions in RKHSs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sepehr Dehdashtian, Lan Wang, Vishnu Naresh Boddeti
FAIRERCLIP debiases CLIP's zero-shot predictions by transforming image and text representations within Reproducing Kernel Hilbert Spaces, effectively mitigating both intrinsic dependencies and spurious correlations. The approach significantly reduces bias metrics, such as achieving an Equal Opportunity Difference of 0.005% on CelebA for intrinsic dependency, and demonstrates up to 100x faster training times compared to existing methods.
Key insights
- Formulating debiasing within RKHSs allows for modeling complex non-linear dependencies and yields highly efficient, closed-form solutions, substantially accelerating the debiasing process.
- A single, generalized objective function using HSIC can effectively address both intrinsic dependencies (where target and sensitive attributes are inherently linked) and spurious correlations, providing a versatile debiasing framework.
- Iterative refinement of pseudo-labels for target attributes, while keeping sensitive attribute pseudo-labels fixed, enables robust and effective debiasing even in scenarios with limited or no ground-truth labels.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.