Reproduce “Group Sparse Regularization for Deep Neural Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Simone Scardapane, Danilo Comminiello, Amir Hussain, Aurelio Uncini
The paper extends group Lasso regularization to deep neural networks, enabling simultaneous input feature selection and hidden neuron pruning within a single training framework. This approach leads to significantly more compact network architectures with comparable predictive performance across various classification tasks.
Key insights
- By penalizing groups of outgoing weights from a unit, structured sparsity is achieved, allowing for the complete removal of entire input features or hidden neurons.
- A single, integrated optimization objective can simultaneously perform weight optimization, neuron pruning, and feature selection, streamlining the deep learning model design process.
- Combining group-level and individual weight-level sparsity via Sparse Group Lasso yields the most effective network compression while maintaining competitive predictive performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.