Reproduce “Synthetically Generating Human-like Data for Sequential Decision Making Tasks via Reward-Shaped Imitation Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bryan Brandt, Prithviraj Dasgupta
Researchers at the Naval Research Laboratory developed a technique to synthesize human-like sequential decision-making data using reward-shaped imitation learning, effectively generating low-divergence trajectories from just five human demonstrations and outperforming a pure DQN agent in complex environments, with METEOR scores up to 0.78. This approach addresses the scarcity of human interaction data for training AI in critical domains.
Key insights
- Reward shaping effectively guides Deep Q-Network (DQN) agents to learn task-completing policies even with minimal human demonstration data.
- The DAgger imitation learning algorithm, when using the reward-shaped RL agent as an expert, successfully generalizes human-like behavior.
- Seeding DAgger with initial sparse human data (DAgger+E) further improves the quality and diversity of generated synthetic trajectories, leading to higher similarity to actual human behavior.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.