Reproduce “Spectral Representations for Convolutional Neural Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Oren Rippel, Jasper Snoek, Ryan P. Adams
Researchers from MIT and Harvard present a framework that leverages spectral representations for Convolutional Neural Networks, introducing spectral pooling for improved dimensionality reduction and spectral parametrization for faster training. The methods yield better information preservation in pooling and accelerate network convergence by factors of 2.2 to 5.1.
Key insights
- The spectral domain offers a fundamental representation for CNN design and optimization, extending its utility beyond just accelerating convolution computations.
- Spectral pooling preserves significantly more information and provides continuous control over output dimensionality compared to conventional pooling techniques.
- Optimizing filters in the frequency domain alters the optimization landscape, making it more favorable for adaptive optimizers due to sparser and more localized representations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.