Reproduce “Disentanglement Beyond Generative Models with Riemannian ICA” on OpenResearch

About this paper

Edmond Cunningham

A novel Riemannian ICA framework offers a theoretical basis for interpreting disentangled features in modern representation learning by defining disentanglement as a local geometric property, moving beyond restrictive global generative models. Experiments showed RICA's robust source recovery and coordinate invariance across various manifolds.

Key insights

• Disentanglement can be fundamentally viewed as a local phenomenon, where interpretable factors of variation arise from local invertibility around each data point, rather than requiring a global invertible mapping.
• Riemannian geometry provides the necessary formal machinery, including geodesics and covariant Hessians, to precisely define and operationalize this local perspective of disentanglement.
• The derived disentanglement tensor mathematically connects pointwise statistical independence of latent components to intrinsic geometric properties of the data manifold, incorporating terms like the covariant Hessian of the Hausdorff density and Ricci curvature.

Summary by alphaXiv.