Reproduce “3DPPE: 3D Point Positional Encoding for Multi-Camera 3D Object Detection Transformers” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Changyong Shu, JIajun Deng, Fisher Yu, Yifan Liu

The authors present 3DPPE, estimating each image pixel’s depth and back-projecting it to a 3D point for positional encoding. A shared encoder embeds these points and learnable object-query points in a common space. On nuScenes’ test set, VoVNet-99 3DPPE scores 0.460 mAP and 0.514 NDS, versus PETR’s 0.441 and 0.504; scale, orientation, and attribute errors are higher.

Key insights

  • Precisely localized 3D point information, even when approximated through depth estimation, provides superior spatial context for transformer-based 3D object detection compared to coarse ray-based encodings.
  • A dedicated, lightweight depth estimation module, combining direct and categorical predictions, is effective for obtaining the necessary per-pixel 3D coordinates in camera-only systems.
  • Employing a unified 3D positional embedding space for both image features and object queries enhances their interaction within the transformer decoder, leading to more accurate 3D localization.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.