Reproduce “Trust Region Inverse Reinforcement Learning: Explicit Dual Ascent using Local Policy Updates” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Anish Diwan, Davide Tateo, Christopher E. Mower, Haitham Bou-Ammar, Jan Peters, Oleg Arenz

TRIRL learns rewards from expert demonstrations while avoiding a full reinforcement-learning solve after every reward update. It makes a trust-region-limited policy update, then corrects the reward step so the updated policy is optimal for that smaller change under the theory’s assumptions. Across imitation-learning tasks, its max-η variant reports normalized IQM 0.7881, versus GAIL’s 0.3297, with function approximation limiting guarantees.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.