Reproduce “Instructions Before Actions: Prospective and Retrospective Language Supervision for Driving Vision-Language-Action Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Angel Martinez-Sanchez, Ross Greer
The authors introduce ADVLAT-Engine, which aligns spoken driving instructions with video and vehicle trajectories, and test four driving models on passenger drives. Fine-tuning on navigation-app drives lowers average displacement error by 23–59%, but instruction effects are inconsistent. At fixed recorded intersections, alternative-maneuver tests reveal stronger response to discrete commands than sentences; these open-loop results do not establish sustained compliance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.