Reproduce “HSTR-Net: Reference Based Video Super-resolution with Dual Cameras” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
H. Umut Suluhan, Abdullah Enes Doruk, Hasan F. Ates, Bahadir K. Gunturk
HSTR-Net, a deep learning architecture developed by researchers at the University of Arizona and Turkish universities, generates high spatio-temporal resolution (HSTR) video by fusing low frame rate high spatial resolution and high frame rate low spatial resolution video feeds from dual-camera systems. It achieves real-time performance on embedded platforms, outperforming existing reference-based super-resolution methods in accuracy and efficiency on aerial and surveillance datasets.
Key insights
- Efficient local attention for patch matching significantly reduces computational complexity compared to exhaustive global methods, making real-time RefSR feasible on embedded systems.
- Combining robust motion estimation with adaptive feature alignment via deformable convolutions is critical for handling complex, dynamic scenarios and misalignments in dual-camera setups.
- A sophisticated fusion module, a modified UNet with residual connections, can effectively reconstruct high-fidelity HSTR video by optimally combining warped references, LR frames, and contextual/patch-matching features.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.