Reproduce “Conformalized Interactive Imitation Learning: Handling Expert Shift and Intermittent Feedback” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Michelle Zhao, Reid Simmons, Henny Admoni, Aaditya Ramdas, Andrea Bajcsy
The ConformalDAgger framework enables robots to adaptively quantify uncertainty and strategically query human experts during interactive imitation learning, even with intermittent feedback. Utilizing the new Intermittent Quantile Tracking (IQT) algorithm, this approach helps robots detect distribution shifts and quickly adapt policies, resulting in increased expert feedback requests (e.g., from ~20% to ~60% in simulation upon expert shift) and improved learning efficiency in simulated and hardware tasks.
Key insights
- Intermittent Quantile Tracking (IQT) maintains asymptotic coverage guarantees for uncertainty estimates even when ground-truth labels are observed probabilistically or intermittently, providing theoretical reliability for online interactive learning.
- Calibrated uncertainty estimates derived from online conformal prediction can effectively detect distribution shifts, particularly changes in expert policy, serving as a robust signal for adaptively querying human experts.
- Directly incorporating intermittent human feedback into the online uncertainty estimation process allows for real-time adaptation of the robot's confidence, leading to more efficient data collection and faster policy retraining.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.