Reproduce “Partial Information Decomposition for Data Interpretability and Feature Selection” on OpenResearch

About this paper

Charles Westphal, Stephen Hailes, Mirco Musolesi

Researchers from University College London and the University of Bologna developed Partial Information Decomposition of Features (PIDF), a novel framework for data interpretability and feature selection. PIDF quantifies per-feature Mutual Information, Feature-Wise Synergy, and Feature-Wise Redundancy, demonstrating improved capabilities in identifying optimal feature subsets and uncovering complex feature interactions across diverse datasets compared to existing methods.

Key insights

• PIDF directly addresses the 'Redundant Variables,' 'Synergistic Variables,' and 'Minimal Subsets' questions, providing a nuanced understanding of how features interact beyond simple importance scores.
• The approach effectively disentangles complex interactions where features are simultaneously redundant and synergistic, which is crucial for deep data understanding and model transparency.
• By making Partial Information Decomposition scalable, PIDF bridges a critical gap between information-theoretic principles and their practical application in machine learning interpretability and feature engineering.

Summary by alphaXiv.