Reproduce “Burst Image Super-Resolution with Base Frame Selection” on OpenResearch

About this paper

Sanghyun Kim, Min Jung Lee, Woohyeok Kim, Deunsol Jung, Jaesung Rim, Sunghyun Cho, Minsu Cho

Researchers from POSTECH developed a new benchmark dataset (NEBI) and a Frame Selection Network (FSN) to enhance Burst Image Super-Resolution by effectively utilizing non-uniformly exposed image bursts and dynamically choosing the optimal base frame. Integrating FSN consistently improved PSNR and SSIM on both synthetic and real-world datasets, surpassing methods relying on fixed or heuristically selected base frames.

Key insights

• Non-uniformly exposed image bursts can yield super-resolution results comparable to those from optimally exposed uniform bursts, underscoring their practical utility in uncontrolled environments.
• Dynamically learning to select the optimal base frame via FSN is substantially more effective than using fixed frames or heuristic auto-exposure methods.
• The Frame Selection Network benefits from a two-stream architecture that effectively combines appearance degradation features with motion-aware correlation for robust base frame selection.
• A specialized Feature Correlation Module (FCM) for feature correlation in spatio-temporal domains is more effective for motion estimation in RAW burst images than generic optical flow networks often pre-trained on RGB data.

Summary by alphaXiv.