Reproduce “Feedback-efficient Active Preference Learning for Socially Aware Robot Navigation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ruiqi Wang, Weizheng Wang, Byung-Cheol Min
The FAPL framework enables socially aware robot navigation by efficiently learning a human-centered reward model from active preferences. This method integrates curious exploration and expert demonstrations to reduce human feedback requirements and results in robot behaviors perceived as more comfortable and natural by humans in both simulated and real-world environments.
Key insights
- Directly distilling human comfort and expectations into a reward model through preferences offers a robust alternative to complex reward engineering or extensive demonstrations.
- Combining diverse robotic exploration with human expert demonstrations significantly improves the feedback and sample efficiency of active preference learning, accelerating convergence.
- Selecting preference queries based on maximum uncertainty effectively minimizes the human workload while maximizing the informativeness of each piece of feedback.
- Continuously re-labeling collected experiences with an evolving reward model is crucial for stable and effective off-policy learning in dynamic human-in-the-loop systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.