Reproduce “Seeing Is Not Knowing: The Epistemic Limits of LLM-Based Mental Health Inference from Youth Social Media Conversations” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xiaohan Ding, Mengting Tu, Andrew H. Zhao, Jamie Zelazny, Candice L. Biernesser, Munmun De Choudhury

The authors tested ten locally deployed reasoning models on Instagram messages donated by 80 adolescents, predicting eight past-month self-report measures. Probe-based retrieval improved item-level exact-response accuracy by a mean of 3.2 percentage points, but messages alone did not establish felt experience. For perceived belonging and connection, score closeness was 76.1%, yet same-band agreement was 30.6% and correlation was .05.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.