Reproduce “Transferable Reward Learning by Dynamics-Agnostic Discriminator Ensemble” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Fan-Ming Luo, Xingchen Cao, Rong-Jun Qin, Yang Yu
A method called Dynamics-Agnostic Discriminator-Ensemble Reward Learning (DARL) learns state-action reward functions from expert demonstrations that remain robust and transferable across environments with varying dynamics. By employing mutual information minimization and a discriminator ensemble, DARL effectively mitigates dynamics and policy dependency in adversarial imitation learning, demonstrating superior performance and high reward consistency across diverse MuJoCo transfer tasks.
Key insights
- Explicitly decoupling the learned reward from environment dynamics through mutual information minimization between state-action representations and future states is crucial for transferable reward functions.
- Employing an ensemble of historical discriminators effectively mitigates policy dependency, allowing the learned reward to accurately guide policy learning from diverse starting points without catastrophic forgetting.
- Transferable state-action dependent reward functions can be effectively learned, challenging the assumption that only state-only rewards are transferable, especially in tasks with significant action dependencies.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.