Reproduce “LU-Net: Invertible Neural Networks Based on Matrix Factorization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Robin Chan, Sarina Penquitt, Hanno Gottschalk
Researchers from Bielefeld University and the University of Wuppertal introduce LU-Net, an invertible neural network architecture based on LU matrix factorization, which significantly reduces computational complexity for inversion and determinant calculation compared to conventional invertible networks. The model achieves competitive density estimation on image datasets while demonstrating substantial gains in training speed and GPU memory efficiency.
Key insights
- Employing LU factorization directly within network layers drastically reduces the computational complexity for inversion from O(D³) to O(D²) and for determinant calculation from O(D³) to O(D).
- A custom-designed 'leaky softplus' activation function can effectively address common challenges in invertible networks, such as bijectivity requirements and gradient stability.
- Maintaining invertibility through solving linear systems for the backward pass offers a numerically stable and efficient alternative to direct matrix inversion.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.