Reproduce “SocialRL: Refining LLMs' Social Intelligence through Multi-turn Reinforcement Learning and Reward Design” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jianing Wang, Xintao Wang, Aili Chen, Jie Shi, Hongcheng Guo, Jun Gao, Wenxuan Zhao, Chengkun Lang and 2 more
SocialRL trains language models for extended social interaction by combining multi-turn PPO with dense, rubric-based rewards and stage-dependent weighting of goal pursuit, strategy, rapport, persona, coherence, and turn quality. Across four social-agent benchmarks, it improves goal achievement by an average of 9.2 percentage points over corresponding base models, while audited process rewards correlate 0.816 with human ratings.
Key insights
- State-dependent value estimates make PPO better suited than the evaluated GRPO variant for conversations in which similarly timed turns can have very different future consequences.
- Process rewards are essential: removing them reduces performance close to the base-model level, while goal advancement and strategic positioning are the most influential dimensions for goal success.
- Relational attunement primarily affects relationship outcomes, showing that optimizing social interaction requires separating task progress from rapport preservation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.