Reproduce “Scaling Therapy Supervision: Assessing Psychotherapy Fidelity Using Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Arvind Pillai, Alexis Roach, Jason Holden, Varsha D Badal, Subigya Nepal, Yu Yvonne Wu, Yuliang Chen, Sudarshan Regmi and 6 more
The authors evaluate a language model system that rates therapist fidelity from de-identified transcripts of real psychotherapy sessions and generates feedback. It scores a 60-item rubric, first locating transcript evidence and then judging each category. On 30 expert-rated sessions, Qwen3.5-9B reached balanced accuracy of 0.72 and κ of 0.46. These results support supervision assistance, not replacement of expert judgment.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.