Reproduce “Revisiting Shape from Polarization in the Era of Vision Foundation Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chenhao Li, Taishi Ono, Takeshi Uemori, Yusuke Moriuchi
Researchers at Sony Semiconductor Solutions Corporation developed a method for single-shot surface normal estimation that effectively leverages polarization cues, outperforming current RGB-only Vision Foundation Models (VFMs). Their approach reduces the Mean Angular Error by 21% over prior SfP methods and 8% over leading RGB-only VFMs, while requiring 33 times less training data and 8 times fewer model parameters.
Key insights
- Polarization cues are a highly efficient modality, enabling state-of-the-art normal estimation with significantly less training data and fewer model parameters compared to RGB-only VFMs.
- Accurately simulating real-world polarization sensor noise at the raw polarized image level is essential to bridge the synthetic-to-real domain gap.
- Integrating strong learned visual priors from pre-trained Vision Foundation Models (like DINOv3) with physics-based polarization features improves model generalization, especially to unseen objects.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.