Reproduce “UniFS: Unified Fast-to-Slow Hierarchical Architecture for Vision-Language-Action Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lin Sun, Zhiwei Guan, Conglin Wang, Zihong Chen, Jianhai Yu, Zongsheng Li, Boyong He, Tao Sun and 2 more
A new architecture, UniFS, unifies fast and slow dynamics within Vision-Language-Action (VLA) models, achieving a 2.1x speedup in inference latency to 17.8 ms and an average 98.3% success rate on the LIBERO benchmark. It generates multi-timescale latent representations directly from the VLM, allowing for adaptive and efficient information transfer to the action expert.
Key insights
- VLM latent representations inherently possess multi-timescale dynamics, with shallower layers capturing faster changes and deeper layers more stable semantic context.
- Effective alignment of these multi-scale latent vectors with an action expert, as achieved by Latent Vector Inversion, is essential to simultaneously boost performance and computational efficiency.
- A unified architecture can resolve the "frequency dilemma" by adaptively generating and utilizing multi-timescale latent representations, providing both fine-grained control and stable contextual information.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.