Reproduce “Afford-VLA: Action-Aligned Visual Planning via Internalized Affordance” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Runze Wang, Yuqian Fu, Yu Li, Tao Lin, Tianwen Qian, Mohamed Elhoseiny, Bo Zhao, Yanwei Fu and 2 more
Afford-VLA, a framework for robot manipulation from Fudan University and collaborators, enhances spatial reasoning in Vision-Language-Action (VLA) models by internalizing task-conditioned affordance as an action-aligned visual planning interface, achieving a 97.4% average success rate on LIBERO and improved zero-shot robustness.
Key insights
- Internalizing affordance as a direct action-aligned visual planning interface significantly enhances the spatial reasoning and precision of VLA models for robot manipulation.
- A tightly coupled perception-action pathway, enabled by internal affordance generation and an action-aligned training strategy, is crucial for effective visual planning and robust robot control.
- A two-stage training approach, starting with ground-truth affordance warm-up and transitioning to model-predicted affordance with ST pooling, effectively aligns the visual planning with downstream action generation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.