Reproduce “Demonstration-Free Robotic Control via LLM Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Brian Y. Tsui, Alan Y. Fang, Tiffany J. Hwu
FAEA (Frontier Agent as Embodied Agent), a framework from independent researchers, applies unmodified general-purpose LLM agent frameworks to robotic manipulation, enabling demonstration-free strategy discovery through iterative program synthesis. It achieved success rates up to 96% on MetaWorld and 88.2% on LIBERO, positioning it as a method for automated training data augmentation for VLA models.
Key insights
- General-purpose LLM agent frameworks, originally developed for software engineering, can effectively discover manipulation programs in embodied environments without requiring demonstrations.
- Robotic manipulation can be framed as an iterative program synthesis and debugging problem, where LLM agents refine strategies in-context through trial and error.
- The use of privileged ground-truth state observations allows the LLM agent to focus on strategic policy discovery, providing a method for generating synthetic, successful trajectories for data augmentation in VLA models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.