Reproduce “Representation Learning via Manifold Flattening and Reconstruction” on OpenResearch

About this paper

Michael Psenka, Druv Pai, Vishal Raman, Shankar Sastry, Yi Ma

FlatNet, a "white-box" approach for representation learning, constructs explicit encoder-decoder neural networks by iteratively flattening an embedded data manifold based on differential geometric principles. It automatically estimates the intrinsic dimension and learns structured latent representations, demonstrating high-quality reconstruction and semantically meaningful interpolations on synthetic datasets and MNIST.

Key insights

• Deep network architectures and parameters can be derived from differential geometric principles, enabling 'white-box' model design with theoretical interpretability.
• Combining local geometric approximations (tangent space, second fundamental form) with global blending functions (partition of unity) effectively constructs layer-wise encoder-decoder maps.
• The 'flattenability' assumption is fundamental to the approach, and while it introduces limitations, it can be addressed by ambient space augmentation for complex manifolds.

Summary by alphaXiv.