Reproduce “Say the Same, Act Differently: Diagnosing and Steering Text-Action Divergence in Reasoning VLAs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zihao Feng, Qingzhao Zhang, Chunyu Xia, Bo Yu, Z. Morley Mao, Ramesh Govindan
The authors identify a hidden-state direction in reasoning VLAs that changes actions while minimally affecting text, then guide projected-gradient attacks toward it. In Alpamayo, the attack preserved the rationale on 89.5% of 200 clips and produced open-loop collisions on 52.6% of 154 scenes. Collision rates did not differ significantly from text-preserving attacks; this is a digital, not physical, attack result.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.