Reproduce “Observing and Controlling Features in Vision-Language-Action Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hugo Buurmeijer, Carmen Amo Alonso, Aiden Swann, Marco Pavone
The authors introduce an observer-controller that reads a frozen vision-language-action model’s internal representation and applies a minimal, input-dependent correction when a predicted feature falls outside a target interval. On DROID hardware, it increased preferred mug-handle grasps from 14% to 74% across 50 rollouts per method. All methods grasped the mug, so this changed preference alignment, not task success or safety.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.