Reproduce “Elastic Queries Reinforcement Learning: Self-Aware Policy Execution for VLA Models” on OpenResearch

About this paper

Ge Wang, Xinyu Tan, Xiang Li, Man Luo, Chengsi Yao, Shenhao Yan, Jiahao Yang, Fan Feng and 6 more

EQRL introduces an adaptive execution framework for Vision-Language-Action (VLA) models, enabling policies to dynamically adjust denoising budgets and action chunk lengths based on perceived state difficulty. This approach reduces amortized inference cost by 20-30% across simulation and real-world robot manipulation tasks, often while maintaining or improving task success rates.

Key insights

• Dynamically adjusting both the denoising budget and action chunk length for VLA models based on task state can lead to more efficient and effective robot manipulation.
• Disagreement among a critic ensemble serves as an effective, learned proxy for state difficulty, guiding adaptive allocation of computational resources.
• Framing variable chunk execution as macro-action reinforcement learning enables a policy to optimize both behavior and execution schedule simultaneously.

Summary by alphaXiv.