Reproduce “Optical Flow Estimation using a Spatial Pyramid Network” on OpenResearch

About this paper

Anurag Ranjan, Michael J. Black

Researchers at the Max Planck Institute developed SPyNet, a Spatial Pyramid Network that integrates a classical coarse-to-fine approach with deep learning to estimate optical flow. SPyNet achieves superior or comparable accuracy to prior deep learning methods on benchmarks like Sintel and KITTI, while being 96% smaller and faster, enabling real-time operation on resource-constrained devices.

Key insights

• Strategic integration of classical coarse-to-fine processing with deep learning effectively addresses the limitations of both purely classical methods (assumption-bound) and purely deep learning approaches (struggle with large motions).
• Breaking down the complex problem of large flow estimation into a series of small residual updates at different scales allows for the use of compact CNNs, leading to a substantial 96% reduction in model size compared to FlowNet.
• The learned spatio-temporal filters at each pyramid level visually resemble classical derivative and Gabor filters, providing interpretability into the network's function, unlike the often 'random-looking' filters in prior end-to-end deep learning models.

Summary by alphaXiv.