Reproduce “ExACT: An End-to-End Autonomous Excavator System Using Action Chunking With Transformers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Liangliang Chen, Shiyu Jin, Haoyu Wang, Liangjun Zhang
Baidu Research developed ExACT, an end-to-end autonomous excavator system that leverages Action Chunking with Transformers (ACT) and multi-modal sensor inputs to directly generate control commands. The system learned simple tasks via direct valve state control and complex multi-stage tasks via joint position control in a simulated environment, requiring only a minimal set of human demonstrations.
Key insights
- End-to-end imitation learning with multi-modal sensor fusion offers a viable path to bypass complex physical modeling for autonomous heavy equipment, like excavators.
- Advanced deep learning architectures like Action Chunking with Transformers (ACT) can learn robust and generalizable excavation policies from a minimal set of human demonstrations.
- For highly dynamic and precise movements, learning direct low-level actuator (valve state) control is challenging; however, kinematic (joint position) control facilitates successful completion of complex multi-stage tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.