Reproduce “NoVo: Norm Voting off Hallucinations with Attention Heads in Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zheng Yi Ho, Siyuan Liang, Sen Zhang, Yibing Zhan, Dacheng Tao
NOVO introduces a lightweight, zero-shot method for mitigating hallucinations in Large Language Models by leveraging the L2 norms of attention heads. This approach establishes a new state-of-the-art accuracy of 78.09% on the TruthfulQA MC1 benchmark and generalizes effectively across over 90% of diverse datasets.
Key insights
- Attention head norms, specifically at the final sequence position, encode valuable and readily accessible signals related to factual truth within LLMs.
- Truth-correlated attention heads, acting as 'Voters,' can be categorized into two types: Type-1 focusing on overall sequence structure and Type-2 on fine-grained token associations, both contributing distinctly to factual predictions.
- Aggregating predictions from a diverse ensemble of these truth-correlated attention heads significantly boosts overall factual accuracy, demonstrating the power of a collective decision over individual model outputs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.