Reproduce “SCALE: Self-uncertainty Conditioned Adaptive Looking and Execution for Vision-Language-Action Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hyeonbeom Choi, Daechul Ahn, Youhan Lee, Taewook Kang, Seongwon Cho, Jonghyun Choi
SCALE turns a VLA model’s own token-level uncertainty into a real-time control signal for both action sampling and visual attention. Without retraining, auxiliary verifiers, or multiple rollouts, it improves performance across simulated and real robotic manipulation tasks, including OpenVLA’s LIBERO-Long success rate from 52.7% to 63.3% and real-world OpenVLA performance from 36.1% to 55.6% on in-distribution tasks.
Key insights
- Perception and action can be coupled at inference time: uncertainty should determine not only whether to explore alternative actions, but also whether to reconsider the visual evidence used for those actions.
- Adaptive action decoding and adaptive visual attention provide complementary benefits; on LIBERO-Long with OpenVLA, their combination reaches 63.3%, versus 58.0% and 56.0% when used separately.
- Using changes in uncertainty relative to recent history is more effective than using instantaneous uncertainty for visual modulation, while a two-pass oracle offers only a small additional gain of 1.3 percentage points.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.