Reproduce “The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Noah Y. Siegel, Oana-Maria Camburu, Nicolas Heess, Maria Perez-Ortiz
The authors introduce Correlational Explanatory Faithfulness (CEF), measuring whether explanations mention input changes in proportion to their effects on model predictions. Its Correlational Counterfactual Test (CCT) correlates intervention total variation distance with a binary mention measure. For Llama-2 70B in predict-then-explain mode, CCT is 0.411 on e-SNLI and 0.083 on ECQA, indicating task-dependent association, not complete faithfulness.
Key insights
- Faithfulness is more accurately captured by a continuous, correlational measure of input impact and explanation content, moving beyond simplistic binary presence checks.
- The proposed CCT can distinguish between explanations that merely mention relevant words and those that strategically mention *impactful* words, providing a more truthful understanding of LLM reasoning behavior.
- Integrating probabilistic prediction changes into intervention-based tests offers a more nuanced diagnostic tool, highlighting explanation behaviors (e.g., verbosity) missed by previous metrics.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.