Reproduce “Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Sébastien Lachapelle, Divyat Mahajan, Ioannis Mitliagkas, Simon Lacoste-Julien

Additive decoder structure can make latent blocks identifiable and support compositional generation beyond the observed training support, even when latent factors are dependent. Theoretical guarantees and synthetic two-object experiments show stronger disentanglement and Cartesian-product extrapolation than comparable non-additive decoders, with LMS scores reaching 99.9 versus 78.1 on dependent block latents.

Key insights

  • Additivity constrains the learned latent transformation through first- and second-order derivatives, producing a block-permutation Jacobian rather than requiring statistically independent latent factors.
  • Identifiability and extrapolation are connected but distinct: block separation helps organize factors, while additivity additionally constrains how unseen combinations are decoded.
  • Global guarantees depend on structural conditions such as path-connected latent support and injective block decoders; disconnected supports can permit different transformations in different regions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.