Reproduce “GVT3D: Gesture-Vision-Text Token Enrichment for 3D Embodied Reference Understanding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Vimarsh Shah, Dulanga Weerakoon, Ajinkya Deshpande, Saksham Agarwal, Nilesh Bhatia, Neel Kiran Naik, Parth Shah, Sravan Danda and 4 more
GVT3D grounds objects in 3D from a person’s language, pose, and a point-cloud scene. Its Embodied Fusion module lets language, gesture, and visual features interact before proposal matching, while foveation focuses depth projection and CLIP processing on selected proposals. On ImputeRefer, it reaches 66.7% overall [email protected] accuracy versus 58.7% for Ges3ViG; the benchmark uses imputed avatars and single-object references.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.