Reproduce “DepressionAgent: Reading, Listening, Seeing, and Deliberating Multimodal Evidence for Depression Risk Assessment” on OpenResearch

About this paper

Fangjie Zhu, Haifeng Lu, Sicheng Zhao, Runhao Zeng, Xiping Hu

DepressionAgent, an agentic framework, shifts multimodal depression risk assessment from implicit feature fusion to explicit evidence deliberation. It achieves 95.69% F1 on D-Vlog and 83.04% F1 on LMVD in a zero-shot setting by systematically observing, deliberating, arbitrating, and reassessing multimodal evidence.

Key insights

• Transforming multimodal depression risk assessment into an explicit, evidence-centric deliberation process significantly enhances the interpretability and robustness of predictions by making the reasoning path inspectable.
• The systematic integration of reflection mechanisms, such as 'Conflict Reflection' for cross-modal disagreements and 'Risk Reflection' for low-risk textual reassessment, effectively corrects erroneous predictions and improves overall assessment accuracy.
• While textual self-report provides strong predictive signals, complementary behavioral cues from audio and visual modalities, when explicitly arbitrated, contribute to a more nuanced and accurate multimodal risk assessment.

Summary by alphaXiv.