Reproduce “Stereo Matching in Time: 100+ FPS Video Stereo Matching for Extended Reality” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ziang Cheng, Jiayu Yang, Hongdong Li
Researchers at Tencent XR Vision Labs and the Australian National University introduce XR-Stereo, a new synthetic indoor video stereo dataset with photorealistic renderings and realistic 6-DoF camera motion. They also propose a continuous video stereo matching framework that leverages temporal redundancy to achieve over 100 FPS inference speed on desktop GPUs and 30 FPS on HMDs while maintaining high accuracy.
Key insights
- Leveraging temporal redundancy by warping and reusing previous frame information allows for a substantial reduction in per-frame computational cost while maintaining or improving disparity estimation accuracy.
- Physically-based rendering, realistic lens effects, and HMD-captured camera motion are crucial for synthetic datasets to bridge the reality gap and effectively train models for real-world XR applications.
- The proposed framework can achieve real-time performance (over 100 FPS) with only a single GRU iteration per frame, comparable to single-frame methods running 20 iterations, demonstrating significant efficiency gains.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.