Reproduce “Nonlinear Concept Erasure: a Density Matching Approach” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Antoine Saillenfest, Pirmin Lemberger
LEOPARD, a method for nonlinear concept erasure, employs orthogonal projections and a density-matching objective to remove sensitive information from neural representations. This approach effectively reduces bias in downstream deep nonlinear classifiers, achieving competitive erasure while better preserving semantic utility, as demonstrated by higher WordSim-353 scores (e.g., 0.64 vs. 0.57 for GLOVE) compared to prior nonlinear methods.
Key insights
- Using linear, orthogonal, non-invertible projections effectively preserves the local geometry of embeddings, crucial for utility, while enabling concept erasure.
- Aligning class-conditional distributions via a density-matching objective (MMD) can achieve robust nonlinear concept removal.
- The rank of the orthogonal projection provides a direct and interpretable mechanism to control the trade-off between erasure strength and utility preservation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.