Reproduce “LocateAnyMesh: Grounding 3D Mesh Queries in 2D Images with Vision-Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Chris Choy, Zhiding Yu, Jan Kautz

LocateAnyMesh uses a textured 3D mesh as a direct query for locating objects in images, encoding sampled surface points into tokens for a vision-language model. It offers coordinate prediction or selection from image-region proposals. On BOP YCB-V, LAM-Select + PPAT achieves 0.7152 AP versus 0.6384 for LAM-Direct + PPAT; these are test results on identities seen during training.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.