Reproduce “Minding Motivation: The Effect of Intrinsic Motivation on Agent Behaviors” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Leonardo Villalobos-Arias, Grant Forbes, Jianxun Wang, David L Roberts, Arnav Jhala
An empirical study investigated how intrinsic motivation (IM) affects the learned behaviors of reinforcement learning agents, observing that IM consistently alters agent policies despite theoretical guarantees of invariance. The research demonstrated that IM typically improves task performance and convergence, and that behaviors traditionally seen as 'reward hacking' can sometimes lead to more optimal outcomes.
Key insights
- Intrinsic Motivation methods consistently alter the learned policies of RL agents, even those with theoretical guarantees of optimality preservation like GRM, highlighting a practical gap between theory and empirical observation.
- Behaviors traditionally labeled as 'reward hacking' are not always detrimental; IM-induced policy changes can lead to more optimal policies or faster task completion in certain contexts.
- Distinct behavioral archetypes emerge from different IM methods, such as State Count promoting extensive early exploration and Max Entropy favoring focused policy refinement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.