Reproduce “3D Copy-Paste: Physically Plausible Object Insertion for Monocular 3D Detection” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yunhao Ge, Hong-Xing Yu, Cheng Zhao, Yuliang Guo, Xinyu Huang, Liu Ren, Laurent Itti, Jiajun Wu

A new framework, "3D Copy-Paste," enables the physically plausible insertion of 3D objects into indoor scenes to augment datasets for monocular 3D object detection. The method improved ImVoxelNet's mean Average Precision at 0.25 IOU by 2.83 percentage points on the SUN RGB-D dataset and 2.8 percentage points on ScanNet by carefully modeling geometric and photometric properties.

Key insights

  • Physically plausible 3D object insertion, encompassing both accurate geometry and realistic illumination, is crucial for effective data augmentation in 3D object detection tasks.
  • Non-plausible insertions, such as arbitrary object placement or simple camera point lighting, can degrade downstream model performance, highlighting the necessity of careful physical modeling.
  • Spatially-varying dynamic lighting and realistic cast shadows significantly contribute to detection performance gains, demonstrating that photometric realism is as vital as geometric plausibility.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.