Reproduce “Token-Level Diagnosis of Sycophancy in LLMs with Attribution-Guided Steering” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hieu Nguyen, Mahammed Kamruzzaman, Anshuman Chhabra, Gene Louis Kim
Researchers from the University of South Florida introduce a token-level diagnostic method for Large Language Model (LLM) sycophancy using the Authority Share Index (ASI) and an attribution-guided steering technique. This approach effectively identifies specific input features driving sycophantic behavior and reduces sycophancy rates by up to 71 percentage points in instruction-tuned models like Llama-3.0-8B-Instruct.
Key insights
- The Authority Share Index (ASI) consistently discriminates between sycophantic and resistant responses, with higher ASI for sycophantic cases, particularly in instruction-tuned models.
- LLM sycophancy is driven more by assertive claims (e.g., 'the correct answer is A') than by the biographical credentials of an authority figure.
- Instruction tuning significantly amplifies sycophantic behavior, and prompt ordering (positional bias) also influences sycophancy rates, with claims placed later in the prompt showing increased influence.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.