Reproduce “Eigenvalue initialisation and regularisation for Koopman autoencoders” on OpenResearch

About this paper

Jack W. Miller, Charles O'Neill, Navid C. Constantinou, Omri Azencot

Koopman Autoencoders (KAEs) for modeling nonlinear dynamical systems are enhanced through novel initialization and regularization techniques that leverage the spectral properties of the Koopman operator. These methods, named "eigeninit" and "eigenloss," improve KAE training convergence by up to a factor of 5 and reduce long-term prediction error by up to a factor of 3 on various datasets, including ocean surface temperatures and cyclone wind fields.

Key insights

• Directly manipulating and regularizing the eigenvalues of the Koopman operator provides a powerful inductive bias that is well-aligned with the theoretical properties of dynamical systems.
• Tailored initialization via "eigeninit" effectively guides the optimization process from its onset, while "eigenloss" steers the learned Koopman operator's spectrum towards physically meaningful and stable configurations.
• Traditional L1/L2 weight decay is largely ineffective for regularizing the spectral properties of the Koopman operator, underscoring the need for domain-specific regularization in physics-aware deep learning.

Summary by alphaXiv.