Reproduce “Detecting Emotional Dynamic Trajectories: An Evaluation Framework for Emotional Support in Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zhouxing Tan, Ruochong Xiong, Yulong Wan, Jinlong Ma, Hanlin Xue, Qichun Deng, Haifeng Jing, Zhengtong Zhang and 3 more

Researchers from Peking University and Guangzhou Quwan Network Technology developed a user-centered evaluation framework for assessing the long-term emotional support capabilities of large language models. The framework uses dynamic emotional trajectories, psychologically-grounded strategies, and causally-adjusted emotion estimation, revealing disparities among models and identifying effective support strategies.

Key insights

  • Evaluating emotional support in LLMs requires a shift from static, model-centric snapshots to dynamic, user-centered, trajectory-based assessments that prioritize long-term impact on user well-being.
  • Integrating established psychological theories and realistic emotional perturbation events is crucial for developing robust and therapeutically sound AI emotional support systems.
  • Causally-adjusted emotion estimation significantly improves the accuracy and trustworthiness of evaluations by mitigating the influence of unobserved confounding factors on user emotional state tracking.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.