Reproduce “Joint Goal and Strategy Inference across Heterogeneous Demonstrators via Reward Network Distillation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Letian Chen, Rohan Paleja, Muyleng Ghuy, Matthew Gombolay
Researchers at Georgia Institute of Technology developed Multi-Style Reward Distillation (MSRD), an Inverse Reinforcement Learning (IRL) framework that simultaneously infers a shared task reward and individual strategic preferences from heterogeneous human demonstrations. This approach yielded a task reward correlation of 0.998 in simulated tasks and enabled a robot to learn diverse table tennis strokes with an 83% success rate.
Key insights
- Heterogeneous demonstrations, often a challenge for IRL, can be leveraged to distill a more robust and unambiguous underlying task reward by explicitly separating it from diverse strategic preferences.
- By re-parameterizing the combined reward function directly within the discriminator of an adversarial IRL setup, the shared task reward can receive direct gradient updates from all strategies, accelerating learning of the common task objective.
- L2-norm regularization applied to the strategy-only reward components encourages parsimony, ensuring that strategy-specific networks only encode necessary variations and promoting greater knowledge sharing into the shared task reward.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.