Reproduce “Efficient Active Imitation Learning with Random Network Distillation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Emilien Biré, Anthony Kobanda, Ludovic Denoyer, Rémy Portelas

RND-DAgger is an active imitation learning method that leverages Random Network Distillation for state-based out-of-distribution detection to trigger expert interventions. This approach achieves competitive task performance while significantly reducing human expert burden and intervention frequency compared to action-based methods.

Key insights

  • State-based out-of-distribution detection provides a more robust and efficient mechanism for active imitation learning, particularly when interacting with variable human experts, compared to action-based discrepancy measures.
  • Strategically targeted expert interventions, based on the agent's encounter with novel states, can significantly reduce the overall expert burden without compromising the agent's final task performance.
  • Ensuring a minimum duration for expert interventions through a 'Minimal Expert Time' mechanism contributes to a more stable and less disruptive active learning process.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.