Reproduce “CollabBench: Benchmarking and Unleashing Collaborative Ability of LLMs with Diverse Players via Proactive Engagement” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hong Qian, Yuanhao Liu, Zihan Zhou, Zongbao Zhang, Hanjie Ge, Haotian Shi, Liang Dou, Xiangfeng Wang and 2 more
CollabBench introduces a benchmark and training paradigm for enhancing Large Language Models' (LLMs) ability to proactively collaborate with diverse human partners in cooperative game environments. This framework develops agents that balance task efficiency with affective adaptation, demonstrating that models trained with a hybrid reward mechanism achieve improvements in both operational performance and social engagement.
Key insights
- Mainstream LLMs struggle to balance task efficiency with affective capabilities in collaborative settings, often lacking sensitivity to interaction dynamics and partner intentions.
- Incorporating personality profiles and affective rewards during training is crucial for developing agents that are both efficient and affectively aligned with diverse partners.
- The proposed training paradigm effectively reduces token consumption by promoting thought condensation, allowing agents to filter redundant 'over-thinking' for more agile interaction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.