Reproduce “Speech as a Multimodal Digital Phenotype for Multi-Task LLM-based Mental Health Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mai Ali, Christopher Lucasius, Tanmay P. Patel, Madison Aitken, Jacob Vorstman, Peter Szatmari, Marco Battaglia, Deepa Kundur
Researchers at the University of Toronto developed a trimodal, multi-task, and longitudinal framework using speech data to predict adolescent mental health conditions. The system, leveraging a mental health-specific Large Language Model (LLM) and integrating text, acoustic landmarks, and vocal biomarkers, achieved a balanced accuracy of 70.8% for depression detection.
Key insights
- Integrating multiple speech-derived modalities (text, acoustic landmarks, vocal biomarkers) consistently enhances predictive performance for mental health conditions compared to unimodal or bimodal approaches.
- Domain-specific LLMs, such as Mental-Alpaca, substantially outperform general-purpose LLMs like LLaMA-2-7B for mental health prediction tasks, highlighting the value of specialized pre-training.
- Modeling temporal changes across patient interactions through a longitudinal framework and using multi-task learning to account for comorbidity significantly improves the predictive accuracy and robustness of mental health assessments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.