Reproduce “Explicitly disentangling image content from translation and rotation with spatial-VAE” on OpenResearch

About this paper

Tristan Bepler, Ellen D. Zhong, Kotaro Kelley, Edward Brignole, Bonnie Berger

A novel variational autoencoder framework, spatial-VAE, is introduced to explicitly separate image content from pose variables like rotation and translation. This approach improves image reconstruction and accurately recovers underlying semantic content, achieving a 0.95 correlation with ground truth conformational factors while decorrelating from rotation on simulated cryo-EM data.

Key insights

• By applying geometric transformations directly to spatial coordinates within the generative model, the conditional image probability becomes differentiable with respect to pose parameters, enabling their explicit inference.
• Designing a generative model that directly incorporates spatial coordinates allows for explicit disentanglement of specific transformations (rotation, translation) from content, offering more interpretable latent spaces than unconstrained disentanglement methods.
• The framework enables continuous modeling of intrinsic content variations (e.g., protein conformations) by separating them from nuisance pose variables, addressing limitations of discrete approaches.

Summary by alphaXiv.