Reproduce “Language-to-Space Programming for Training-Free 3D Visual Grounding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Boyu Mi, Hanqing Wang, Tai Wang, Yilun Chen, Jiangmiao Pang
The Language-to-Space Programming (LASP) method introduces an approach for training-free 3D visual grounding by employing Large Language Models to generate and optimize executable Python code for spatial relation encoding. It achieves 52.9% accuracy on the Nr3D benchmark, outperforming prior training-free methods, while significantly reducing grounding time and token costs compared to agent-based approaches.
Key insights
- LLMs can effectively program and self-correct executable geometric reasoning code, allowing for more precise and flexible spatial relation encoding than fixed functions or direct LLM reasoning.
- Automated test suites with detailed error messages are crucial for guiding LLM code optimization, leading to robust and accurate spatial relation encoders.
- Executing LLM-generated spatial relation code outside the LLM inference loop significantly reduces computational costs while maintaining high accuracy, achieving a balance between efficiency and performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.