Reproduce “Measuring Depression Symptom Severity from Spoken Language and 3D Facial Expressions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Albert Haque, Michelle Guo, Adam S Miner, Li Fei-Fei
A multi-modal machine learning method, integrating spoken language and 3D facial expressions, measures depression symptom severity with competitive accuracy. This system, utilizing novel sentence-level embeddings and causal convolutional networks, predicted PHQ scores with an average error of 3.67 points and classified major depressive disorder with 83.3% sensitivity and 82.6% specificity.
Key insights
- Combining multi-modal information from audio, visual, and linguistic cues consistently yields superior performance in depression severity assessment compared to unimodal approaches.
- Learned sentence-level embeddings effectively capture comprehensive temporal context, outperforming both traditional hand-crafted features and averaged pre-trained embeddings.
- Causal Convolutional Networks (C-CNNs) provide an efficient deep learning architecture for modeling long, variable-length multi-modal sequences without relying on explicit interview context.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.