Reproduce “Parallax-tolerant Image Stitching via Segmentation-guided Multi-homography Warping” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tianli Liao, Ce Wang, Lei Li, Guangen Liu, Nan Li
An image stitching method uses the Segment Anything Model (SAM) to guide multi-homography warping, which effectively addresses large parallax by enabling content-aware, discontinuous alignment of foreground objects and background scenes. The approach consistently reduces ghosting artifacts and achieves superior quantitative metrics, such as a PSNR of 19.19 on the Parallax dataset, compared to ten other state-of-the-art methods.
Key insights
- Integrating a powerful foundation model like SAM for semantic segmentation provides crucial content-aware guidance for multi-homography fitting, enabling discontinuous warps necessary for large parallax.
- Defining the neighborhood system for the energy-based multi-homography fitting using SAM's semantic segments significantly improves the stability and accuracy of homography estimation by ensuring consistent motion models within objects.
- A robust, multi-model fitting approach combined with an occlusion-aware warping strategy (Error-buffer algorithm) allows for simultaneous and accurate alignment of objects at varying depths, mitigating ghosting artifacts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.