Reproduce “When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hui Lu, Yi Yu, Yiming Yang, Chenyu Yi, Qixin Zhang, Bingquan Shen, Alex C. Kot, Xudong Jiang
UPA-RFAS trains one patch on a surrogate VLA using feature-space perturbation plus attention and semantic objectives, then tests transfer to other policies. On LIBERO, transfer to a simulated OpenVLA-oft-w victim yields 5.75% average task success, versus 98.25% benign and 41.25% for the strongest listed baseline. Physical-robot evidence is a qualitative demonstration across three tasks, not a large-scale evaluation.
Key insights
- The ℓ1 deviation and repulsive contrastive regularization in feature space are critical for achieving strong black-box transferability, with contrastive loss being more impactful than ℓ1 magnitude.
- Robustness-augmented optimization, through a bi-level procedure, enhances the universality of patches by making them less dependent on specific surrogate model properties.
- Directly manipulating cross-modal attention and semantic interpretation via PAD and PSM losses allows the patch to inject misleading signals that perturb instruction-conditioned policies across diverse VLA architectures.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.