Reproduce “Lost in Delusion: Examining LLM Safety Under User Delusions and Distress” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Andrew Aquilina, Chetna Nihalani, Vasudha Varadarajan, Nathan S. Fishbein, Yu-Ru Lin, Maarten Sap
The authors test whether delusional framing changes chatbot safety responses using matched 16-turn simulations with synthetic personas and distress-only controls. Six chatbots were evaluated. For OLMo-3-7B, support-directed responses during the final harm-related phase were 37.2% in distress-only conversations and 8.3% in delusional conversations. These simulated response rates do not establish real-world clinical outcomes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.