Reproduce “Reasoning Elicitation in Language Models via Counterfactual Feedback” on OpenResearch

About this paper

Alihan Hüyük, Xinnuo Xu, Jacqueline Maasch, Aditya V. Nori, Javier González

Researchers from Microsoft Research Cambridge and Harvard University developed fine-tuning methods using counterfactual feedback to enhance causal reasoning in large language models. Their approach, particularly 'causal consistency feedback,' resulted in improved consistency in causal judgments and better inductive generalization across various real-world domains like healthcare and engineering.

Key insights

• Optimizing for causal consistency with paired factual and counterfactual feedback is more effective for eliciting robust causal reasoning in LLMs than optimizing for individual answer correctness.
• Large language models can learn to compose causal knowledge, demonstrating strong inductive generalization (e.g., inferring A→C from A→B and B→C) when fine-tuned with counterfactual feedback.
• Evaluating an LLM's causal reasoning requires unit-dependent consistency metrics, such as Necessity Inconsistency Rate and Sufficiency Inconsistency Rate, which go beyond simple accuracy to capture the coherence of causal judgments.

Summary by alphaXiv.