Reproduce “CRISP: Object Pose and Shape Estimation with Test-Time Adaptation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jingnan Shi, Rajat Talak, Harry Zhang, David Jin, Luca Carlone

CRISP introduces a category-agnostic pipeline for 6D object pose and 3D shape estimation from a single RGB-D image. It demonstrates competitive pose accuracy and improved shape reconstruction, while its CRISP-ST self-training mechanism enables self-supervised adaptation to new domains, enhancing performance on unseen objects without additional labeled data.

Key insights

  • A category-agnostic approach that leverages latent shape codes and unnormalized Pose-Normalized Coordinates can effectively estimate both 6D pose and 3D shape from single RGB-D inputs without explicit object category labels.
  • Optimization-based refinement of initial deep learning outputs significantly improves the accuracy of pose and shape estimates and is crucial for generating reliable pseudo-labels for self-supervised domain adaptation.
  • Maintaining unnormalized Pose-Normalized Coordinates during estimation is critical for preventing scale degeneracies and ensuring the stability and effectiveness of the self-training process in new domains.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.