Reproduce “Reinforcement Learning Interventions on Boundedly Rational Human Agents in Frictionful Tasks” on OpenResearch

About this paper

Eura Nofshin, Siddharth Swaroop, Weiwei Pan, Susan Murphy, Finale Doshi-Velez

Harvard University researchers introduced the Behavior Model Reinforcement Learning (BMRL) framework, which utilizes simplified 'chainworld' models of human behavior to provide personalized and interpretable AI interventions in tasks requiring sustained effort. This framework enables rapid personalization and demonstrates robustness across various model misspecifications, assisting individuals in achieving long-term goals.

Key insights

• The concept of 'AI equivalence' defines two human MDPs as equivalent if they lead to the same optimal AI policies, allowing simpler models like chainworlds to effectively represent complex human behaviors for intervention purposes.
• Chainworlds provide an analytically tractable and interpretable human model that enables efficient inference of individual human parameters and the generation of clear, three-window formatted AI intervention policies.
• AI interventions can effectively target specific maladapted human MDP parameters, such as a myopically low discount factor (γ), extending beyond traditional reward or state manipulations.

Summary by alphaXiv.