Reproduce “Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bichen Wu, Alvin Wan, Xiangyu Yue, Peter Jin, Sicheng Zhao, Noah Golmant, Amir Gholaminejad, Joseph Gonzalez and 1 more
UC Berkeley researchers introduce a "shift" operation that serves as a zero-FLOP, zero-parameter alternative to traditional spatial convolutions, leading to substantial reductions in model size and computational cost while maintaining or improving accuracy across image classification, face verification, and style transfer tasks.
Key insights
- Spatial information aggregation can be effectively achieved through simple, parameter-free, and FLOP-free memory shifts, offering a fundamental alternative to costly spatial convolutions.
- Sandwiching the shift operation between 1x1 convolutions makes the overall module trainable via stochastic gradient descent and invariant to channel permutation within the shift.
- Shift-based models demonstrate greater resilience to severe parameter reduction compared to conventional convolutional networks, maintaining better accuracy under tight constraints.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.