Reproduce “Tracking Equivalent Mechanistic Interpretations Across Neural Networks” on OpenResearch

About this paper

Alan Sun, Mariya Toneva

Researchers at Carnegie Mellon University and Max Planck Institute for Software Systems developed a framework to determine if two neural networks implement the same high-level algorithm without explicit interpretation. Their Congruity algorithm estimates this "interpretive equivalence" by comparing representation similarities across causally-generated model implementations, enabling the distinction of algorithmic types and the reduction of larger models to smaller, equivalent ones.

Key insights

• Evaluating interpretive equivalence implicitly compares interpretations, bypassing the need for explicit, potentially subjective, human-understandable descriptions.
• The framework addresses MI scalability by enabling the reduction of complex models to simpler, interpretively equivalent ones and the decomposition of tasks.
• A theoretical duality exists: higher-level, more compressed interpretations correspond to larger, more dispersed sets of implementations.

Summary by alphaXiv.