Reproduce “From Pixels to Actions--Towards Native Vision-Language-Action Policies” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Weichen Fan, Haiwen Diao, Penghao Wu, Siyuan Hu, Ziqi Huang, Tianqi Liu, Mingze Sun, Ziyi Wang and 9 more
NEO-VLA processes pixels and instruction words in one backbone, allowing a continuous-action expert to read representations from selected layers. In LIBERO, distributed access across the 40-layer backbone averages 90.9% success with the backbone frozen, versus 86.7% for final-layer access; after fine-tuning it reaches 95.3%. Real-world performance remains limited, with 17 successes in 20 trials on one task.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.