Reproduce “Aria Digital Twin: A New Benchmark Dataset for Egocentric 3D Machine Perception” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaqing Pan, Nicholas Charron, Yongqian Yang, Scott Peters, Thomas Whelan, Chen Kong, Omkar Parkhi, Richard Newcombe and 1 more
The Aria Digital Twin (ADT) dataset offers a new benchmark for egocentric 3D machine perception, featuring sub-millimeter accurate, photo-realistic digital twins of indoor environments and objects, along with precisely synchronized 6-DoF poses for the Aria device, objects, and human wearers. Initial evaluations show that state-of-the-art models face challenges when applied to this unique egocentric data, indicating substantial room for algorithmic improvements.
Key insights
- ADT provides a new standard for egocentric 3D datasets, offering unprecedented accuracy and a holistic view of human-object-environment interactions crucial for advanced AR applications.
- Initial benchmarking results reveal a significant domain gap between existing large-scale vision datasets (e.g., COCO, ScanNet) and real-world egocentric AR data, indicating that current state-of-the-art models perform poorly on ADT.
- The dataset's unique pairing of real and perfectly aligned photo-realistic synthetic images creates an ideal resource for developing and evaluating sim-to-real learning techniques, which are vital for scaling AR research.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.