Reproduce “Seeking Interpretability and Explainability in Binary Activated Neural Networks” on OpenResearch

About this paper

Benjamin Leblanc, Pascal Germain

Researchers from Université Laval developed a greedy training algorithm (BGN) for Binary Activated Neural Networks (BANNs), yielding inherently interpretable and high-performing models for tabular regression tasks. The approach produces compact BANNs competitive with traditional interpretable models in accuracy, complemented by an adapted method for multi-level SHAP explanations.

Key insights

• Binary Activated Neural Networks operate by partitioning input space with hyperplanes, and their predictive capacity is largely determined by the first hidden layer's ability to define distinct regions.
• Building compact and interpretable neural networks from scratch through a greedy, constructive approach (ante-hoc design) is more effective than attempting to prune or simplify large, pre-trained 'black box' models.
• The complexity cost for improving performance in BANNs (e.g., by adding neurons) can increase linearly, offering a more manageable trade-off for interpretability compared to exponentially growing decision trees.

Summary by alphaXiv.