Reproduce “DriveEditor: A Unified 3D Information-Guided Framework for Controllable Object Editing in Driving Scenes” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yiyuan Liang, Zhiying Yan, Liqun Chen, Jiahuan Zhou, Luxin Yan, Sheng Zhong, Xu Zou
DriveEditor is a unified 3D information-guided diffusion framework that enables controllable object editing in driving scene videos, performing repositioning, replacement, deletion, and insertion. It achieves precise 3D object control and maintains high-fidelity appearance, generating augmented data that improves 3D object detection performance (NDS 0.581 vs. 0.569) on the nuScenes dataset.
Key insights
- Integrating depth information from 3D bounding box projections into a diffusion model significantly enhances precise 3D positional and orientational control of objects.
- A multi-tiered approach to appearance control, combining both direct image data and extracted semantic features with 3D prior information, is crucial for maintaining object fidelity and consistency across video frames.
- A two-stage training strategy effectively balances learning robust position control from bounding boxes with the benefits of 3D information fusion for improved appearance and consistency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.