Reproduce “Long-Term Memory for VLA-based Agents in Open-World Task Execution” on OpenResearch

About this paper

Xu Huang, Weixin Mao, Yinhao Li, Hua Chen, Jiabao Zhao

Researchers from Nanjing University and LimX Dynamics introduced ChemBot, an AI framework that enables autonomous execution of complex chemical laboratory tasks by integrating a hierarchical planning agent with a progress-aware Vision-Language-Action model and dual-layer memory. The system achieved high task success rates, for example, 95% for heat and dissolution, on a collaborative robot, while ensuring smoother, safer operation through asynchronous inference.

Key insights

• Hierarchical decomposition, supported by a Scene Describer and a reflective Subtask Generator, is critical for robust planning in complex, long-horizon tasks by effectively grounding visual information and self-correcting plans.
• A dual-layer memory architecture significantly enhances agent performance by efficiently managing real-time context, reducing inference overhead, mitigating hallucinations, and facilitating long-term knowledge accumulation.
• Integrating a progress-aware head directly into VLA models enables dynamic, closed-loop execution by providing real-time subtask completion feedback, eliminating reliance on fixed time steps and allowing for adaptive task transitions.
• Decoupling inference from the control loop through asynchronous mechanisms and leveraging future-state predictions is essential for maintaining trajectory smoothness, operational safety, and precision in robotic manipulation.

Summary by alphaXiv.