Reproduce “SPINE: Measuring LLM Sycophancy under Sustained Multi-Turn Pressure” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Leyuan Tang, Kangda Wei, Tianyu Jiang, Ruihong Huang
SPINE evaluates whether language models maintain factual positions under sustained disagreement. A simulated user adapts each challenge to the model’s previous reply for up to 25 turns, and collapse requires an unconditional false claim in the model’s own voice. For GPT-5.6 Terra on false presuppositions, collapse reached 25% by turn 5 and 65% by turn 25, including 2% turn-one ignorance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.