Reproduce “Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology” on OpenResearch

About this paper

Bastian Rieck, Matteo Togninalli, Christian Bock, Michael Moor, Max Horn, Thomas Gumbsch, Karsten Borgwardt

Neural Persistence introduces an algebraic topology-based complexity measure for deep neural networks, quantifying structural properties independently of input data. It serves as an effective early stopping criterion, often enabling earlier training cessation with comparable or improved accuracy and without requiring a validation dataset.

Key insights

• Neural persistence effectively distinguishes between different network states, such as trained (converged), diverging, and randomly initialized networks, by showing distinct NP value regimes.
• Deep learning best practices like batch normalization and dropout consistently lead to higher mean normalized neural persistence, suggesting these techniques create topologically more complex or robust network structures.
• Neural persistence can be used as a competitive early stopping criterion that conserves valuable labeled data by eliminating the need for a separate validation dataset.

Summary by alphaXiv.