Reproduce “Representational Difference Explanations” on OpenResearch

About this paper

Neehar Kondapaneni, Oisin Mac Aodha, Pietro Perona

Representational Difference Explanations (RDX), a novel XAI method, discovers and visualizes subtle differences between two learned model representations by identifying inputs that are considered semantically related by one model but not the other. The method consistently outperforms existing XAI techniques in isolating representational differences across various tasks, including recovering engineered distinctions and revealing unique knowledge encoded in state-of-the-art models like DINOv2 and CLIP-iNat.

Key insights

• Defining concepts by inputs that are uniquely considered semantically related by one model (but not another) provides a direct and effective mechanism for contrasting model representations and uncovering subtle, model-specific knowledge.
• Normalizing distances by nearest neighbor ranks helps to make representational spaces comparable across models, allowing for meaningful identification of relative semantic relationships regardless of absolute embedding scale.
• A locally-biased difference function is crucial for filtering out irrelevant differences between already distant embeddings, ensuring that the explanation focuses on meaningful divergences in how models perceive similar data points.

Summary by alphaXiv.