Reproduce “LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference” on OpenResearch

About this paper

Pingjun Hong, Beiduo Chen, Siyao Peng, Marie-Catherine de Marneffe, Barbara Plank

Researchers from LMU Munich, University of Vienna, and UCLouvain introduce LITEX, a linguistically-informed taxonomy for Natural Language Inference (NLI) explanations, revealing that 61.2% of NLI items exhibit within-label variation where annotators agree on a label but use distinct reasoning. Applying this taxonomy to guide Large Language Models (LLMs) significantly improves the quality of generated NLI explanations, yielding higher lexical and semantic similarity to human references and enhanced semantic coverage.

Key insights

• LITEX effectively captures distinct human reasoning strategies in NLI, demonstrating high inter-annotator agreement (Cohen's = 0.862) and learnability by various language models.
• Within-label variation is a pervasive phenomenon, observed in 61.2% of NLI items, where multiple distinct reasoning types contribute to the same NLI label.
• Guiding LLMs with LITEX categories leads to more concise, human-aligned, and semantically diverse explanations, outperforming baseline and highlight-guided generation methods.

Summary by alphaXiv.