Reproduce “Evaluating Language Model Safety Across Long Adversarial Conversations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Parisa Salmani, Peter R. Lewis
The authors measure safety across persistent conversations pursuing harmful objectives. A shadow user pursues the objective as a classifier labels replies safe or unsafe; the study tests three open-weight models on medical-advice and weapon-construction prompts. Safe-response rates fall from 85–100% on the first turn to 15–44% at depth 101, a proof of concept, not an estimate of real-world harm.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.