Reproduce “Neural Additive Experts: Context-Gated Experts for Controllable Model Additivity” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Guangzhi Xiong, Sanchit Sinha, Aidong Zhang
A new machine learning framework, Neural Additive Experts (NAEs), developed at the University of Virginia, introduces a context-gated mixture-of-experts approach to provide controllable model additivity. This framework achieves competitive predictive accuracy comparable to complex black-box models while preserving transparent, feature-level explanations and offering architectural bounds on feature effects.
Key insights
- Applying a mixture-of-experts paradigm at the feature level with context-aware gating allows models to capture complex, non-additive interactions while preserving the fundamental O(n) additive structure for interpretability.
- A dedicated expert variation penalty provides a mathematical knob to explicitly tune the model's additivity, enabling a continuous spectrum from highly flexible to strictly additive behaviors.
- Architectural bounds for individual feature effects offer robust, model-intrinsic guarantees on the potential range of a feature's contribution, enhancing interpretability beyond empirical variance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.