Reproduce “Causal Micro-Narratives” on OpenResearch

About this paper

Mourad Heddaya, Qingcheng Zeng, Chenhao Tan, Rob Voigt, Alexander Zentefis

Researchers from the University of Chicago and Northwestern University developed a framework to identify and classify "causal micro-narratives"—sentence-level causal explanations specific to a target subject—from text. They trained and evaluated fine-tuned Large Language Models, achieving micro-averaged F1 scores up to 0.71 for multi-class classification of inflation narratives in news articles.

Key insights

• Fine-tuned, smaller LLMs (e.g., Llama 3.1 8B) significantly outperform larger, in-context learning models (e.g., GPT-4o) for this domain-specific, multi-label causal narrative classification task, highlighting the importance of task-specific training.
• Combining both historical and contemporary data during fine-tuning can enhance model performance and generalization capabilities across substantial temporal and linguistic shifts.
• The primary challenge for non-fine-tuned models, such as GPT-4o, is distinguishing non-narrative sentences from causal micro-narratives, indicating a struggle to precisely grasp the task definition from limited examples.

Summary by alphaXiv.