Reproduce “KAE: Kolmogorov-Arnold Auto-Encoder for Representation Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Fangchen Yu, Ruilizhen Hu, Yidong Lin, Yuqi Ma, Zhenghao Huang, Wenye Li
The Kolmogorov-Arnold Auto-Encoder (KAE) integrates Kolmogorov-Arnold Networks into autoencoders by employing learnable polynomial activation functions, achieving superior latent representations and reconstruction quality across various tasks and datasets. This architecture outperforms traditional autoencoders and other KAN variants in representation learning efficiency and performance.
Key insights
- Learnable polynomial activation functions are more stable and effective for integrating KANs into autoencoders than B-splines, providing a better balance between flexibility and stability.
- The adaptive non-linearities of KAE layers allow the model to capture more intricate data relationships, leading to superior latent representations that are robust for various downstream tasks.
- KAE models achieve higher performance with fewer learnable parameters compared to other KAN variants, demonstrating improved parameter efficiency and faster convergence.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.