Reproduce “Exposition and Interpretation of the Topology of Neural Networks” on OpenResearch

About this paper

Rickard Brüel Gabrielsson, Gunnar Carlsson

Applying Topological Data Analysis (TDA) to the spatial filters of Convolutional Neural Networks (CNNs) reveals that learned weights consistently form simple, recurrent topological structures like a "primary circle" representing rotating edges. These topological structures correlate with a network's generalization ability, and incorporating them as input features can accelerate training and improve accuracy.

Key insights

• Spatial filters of CNNs consistently organize into simple, recurrent topological structures, most notably a "primary circle" representing rotating edges, across various datasets and network depths.
• More complex, unanticipated topological structures (e.g., a "strange circle" or a "three-circle model" reminiscent of a Klein bottle) were also identified in specific contexts.
• The strength and persistence of these learned topological structures correlate with a network's ability to generalize to unseen data and even to cross-dataset tasks.

Summary by alphaXiv.