Reproduce “MIS-Bench: Benchmarking Multimodal LLMs for Psychotherapeutic Interpersonal Skills Assessment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuhan Lu, Yi Yao, Hua Shen, Katie Aafjes-van Doorn, Zhaonan Wang
MIS-Bench evaluates whether multimodal models can match trained raters’ rubric-based judgments of therapist responses across eight interpersonal skills. It tests nine models with different input modalities and prompting styles, then introduces MIS-RAFT, which trains scores as numerical values using squared error. GPT-5.4 reaches 38.6% mean ICC, while MIS-RAFT raises Qwen2.5-VL’s mean to 31.1% in a focused comparison, not establishing generalization.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.