Reproduce “Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Huihan Liu, Soroush Nasiriany, Lance Zhang, Zhiyao Bao, Yuke Zhu
Researchers at The University of Texas at Austin developed Sirius, a human-in-the-loop framework for robot learning that enables continuous policy improvement during real-world deployment while ensuring task success through human intervention. The system demonstrated an 8% increase in success rate in simulation and 27% on real hardware compared to state-of-the-art methods, alongside a reduction in human workload.
Key insights
- Human interventions provide critical, high-value signals for learning, implicitly revealing risk-sensitive states and necessary corrective actions for robust policy improvement.
- Explicitly down-weighting robot actions that immediately precede human interventions is crucial to prevent the policy from learning suboptimal behaviors that lead to failure.
- Intelligent memory management, specifically retaining data with more interventions, significantly improves learning efficiency and performance by focusing on critical experiences while reducing memory burden.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.