Reproduce “Transductive Visual Programming: Evolving Tool Libraries from Experience for Spatial Reasoning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shengguang Wu, Xiaohan Wang, Yuhui Zhang, Hao Zhu, Serena Yeung-Levy

Stanford University researchers developed Transductive Visual Programming (TVP), an experience-driven framework for AI agents to automatically evolve their tool libraries for spatial reasoning. The system achieves 33.3% accuracy on the Omni3D-Bench dataset, outperforming previous visual programming methods by 11% and large multimodal models like GPT-4o by 22%, while continuously simplifying programs through validated tool abstraction.

Key insights

  • Grounding tool creation in validated, successful problem-solving experiences (a transductive approach) leads to genuinely useful and frequently utilized abstractions, unlike speculative inductive methods.
  • A continuous, closed-loop cycle of problem-solving and experience-driven tool abstraction enables AI agents to progressively improve their capabilities and simplify complex reasoning tasks.
  • By abstracting multi-step computational logic into higher-level tools, TVP reduces program complexity, increases accuracy, and significantly enhances an agent's ability to tackle precise 3D spatial reasoning problems.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.