Reproduce “Object-level Scene Deocclusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhengzhe Liu, Qing Liu, Chirui Chang, Jianming Zhang, Daniil Pakhomov, Haitian Zheng, Zhe Lin, Daniel Cohen-Or and 1 more
PACO, a self-supervised, parallel visible-to-complete diffusion framework developed by researchers from The Chinese University of Hong Kong, Adobe, and others, accurately reconstructs occluded object parts in real-world images with higher fidelity and mask accuracy than prior methods. It achieved an IoU of 89.52 and an FID of 13.93 on the COCOA validation set, enabling advanced image editing and single-view 3D scene reconstruction.
Key insights
- Leveraging pre-trained diffusion models' rich prior knowledge, adapted with an object-aware framework, can significantly overcome the generative limitations of prior regression-based deocclusion methods.
- Object-level deocclusion is fundamentally distinct from generic image inpainting; it requires specific mechanisms to preserve object identity and predict amodal masks without explicit input.
- A self-supervised approach using synthetically generated, diverse object ensembles can effectively train models for real-world deocclusion, bypassing the need for impossible ground-truth occluded annotations.
- A layer-wise deocclusion strategy efficiently processes multiple objects within the same depth layer, balancing computational cost and output quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.