Reproduce “Topological Approaches to Deep Learning” on OpenResearch

About this paper

Gunnar Carlsson, Rickard Brüel Gabrielsson

This work applies Topological Data Analysis (TDA) to analyze the internal representations learned by deep neural networks and proposes a mathematical formalism for designing architectures that leverage data geometries. The research reveals that initial CNN layers learn features consistent with biological visual processing and demonstrates that incorporating feature geometry can improve training speed by up to 3.5x and enhance generalization on image datasets.

Key insights

• Initial layers of convolutional neural networks (CNNs) learn topological structures, such as a circular manifold for oriented edges, consistent with natural image statistics and the mammalian visual pathway.
• Deep CNNs exhibit hierarchical feature abstraction, with higher layers learning more complex and diffuse topological structures beyond simple oriented features.
• Explicitly incorporating geometric and topological properties of data features into neural network architectures can improve training efficiency and generalization capabilities.

Summary by alphaXiv.