Reproduce “PromptStereo: Zero-Shot Stereo Matching via Structure and Motion Prompts” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xianqi Wang, Hao Yang, Hangtian Wang, Junda Cheng, Gangwei Xu, Min Lin, Xin Yang
PromptStereo enhances zero-shot stereo matching by introducing a Prompt Recurrent Unit (PRU) that integrates monocular depth foundation model decoders and structure/motion prompts into the iterative refinement process. The method achieves state-of-the-art generalization across diverse benchmarks, including challenging scenes with reflective surfaces, while maintaining efficiency.
Key insights
- Directly leveraging the pre-trained decoder of a monocular depth foundation model within the recurrent unit significantly embeds powerful priors into the refinement loop for better generalization.
- A simple residual prompting mechanism effectively guides the iterative refinement process without distorting the inherited, rich representations from the foundation model.
- Affine-invariant normalization and fusion of initial disparities provide a globally consistent and reliable starting point, improving overall geometric accuracy.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.