Reproduce “Risk-Aware Preference Learning for Stochastic Outcomes” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yi-Shiuan Tung, Yuni Wu, Wei Jiang, Alessandro Roncone, Bradley Hayes
Researchers at the University of Colorado Boulder investigated whether explicitly modeling human risk sensitivity through Cumulative Prospect Theory (CPT) improves reward function recovery from human preferences over stochastic robot outcomes. Their framework demonstrates that a CPT-aware learner significantly outperforms an Expected Utility (EU) learner when human preferences exhibit risk-sensitive biases, leading to more accurate reward functions.
Key insights
- A mismatch between the learner's assumed preference model and the actual human decision-making process (e.g., EU learner with CPT human) leads to suboptimal reward recovery.
- The CPT learner can effectively disentangle what users truly value (reward weights) from how they perceive and weight uncertainty (risk sensitivity parameters).
- Explicitly modeling human risk sensitivity, particularly through CPT, is crucial for accurate reward learning when human preferences are formed under uncertainty.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.