Reproduce “From Loops to Oops: Fallback Behaviors of Language Models Under Uncertainty” on OpenResearch

About this paper

Maor Ivgi, Ori Yoran, Jonathan Berant, Mor Geva

Researchers at Tel Aviv University characterized Large Language Model failures like repetitions, degenerate text, and hallucinations as a consistent hierarchy of "fallback behaviors" under epistemic uncertainty. They demonstrated that "strengthening" LLMs through scaling, pretraining, or instruction-tuning shifts these models towards generating more subtle and harder-to-detect hallucinations, a trend also exacerbated by common decoding methods.

Key insights

• LLM fallback behaviors exhibit a consistent hierarchical order under uncertainty: repetitions → degenerate text → hallucinations.
• As LLMs become 'stronger' (more parameters, longer pretraining, instruction-tuned), they shift from simpler, easily detectable repetitions to more complex and subtle hallucinations.
• Common random sampling decoding techniques (e.g., higher temperature) reduce repetitions but critically increase the rate of hallucinations, indicating a trade-off in error types.
• LLMs consistently fail to proactively abstain from generating content even when explicitly prompted or faced with clear lack of knowledge, opting to fabricate instead.

Summary by alphaXiv.