Reproduce “Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data” on OpenResearch

About this paper

Paul Pu Liang, Terrance Liu, Anna Cai, Michal Muszynski, Ryo Ishii, Nicholas Allen, Randy Auerbach, David Brent and 2 more

Researchers from Carnegie Mellon University, in collaboration with clinical institutions, developed a method to predict daily mood in high-risk adolescents using multimodal smartphone data while preserving user privacy. Their approach demonstrates that combining typed text, keystroke timings, and app usage can effectively infer mood, and a novel privacy-preserving technique reduces user identity leakage from learned representations by approximately half.

Key insights

• Multimodal smartphone data, including typed text, keystrokes, and app usage, are individually and collectively predictive of daily mood in high-risk adolescents.
• Keystroke dynamics provide unique contextual information that enhances mood prediction when fused with typed text, demonstrating the value of fine-grained behavioral signals.
• Simple bag-of-words models can outperform complex pretrained language models on informal, long-context typed mobile data, suggesting challenges for current large LMs in this domain.
• Standard machine learning models trained on sensitive mobile data can inadvertently encode and reveal user identity, necessitating dedicated privacy-preserving techniques.

Summary by alphaXiv.