Reproduce “The Curious Case of Factual (Mis)Alignment between LLMs' Short- and Long-Form Answers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Saad Obaid ul Islam, Anne Lauscher, Goran Glavaš
A new framework, SLAQ, evaluates Large Language Models' factual consistency by comparing responses to identical questions posed in isolation versus embedded in multi-fact queries. Findings indicate LLMs often exhibit high raw alignment but are more consistently incorrect than correct, and that underlying mechanistic similarities strongly predict this behavioral alignment.
Key insights
- LLMs exhibit high raw factual alignment (73-78%) across query complexities, but this is predominantly driven by systematic failures (being incorrect in both formats) rather than systematic successes.
- Momentum effects are observed in long-form answers; preceding correct answers increase the probability of subsequent facts being correct, while consecutive errors lead to a decrease in accuracy.
- Factual alignment at the behavioral level directly correlates with mechanistic similarity in the internal component activation pathways (attention heads and MLP layers) within LLMs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.