Reproduce “Domain-Specific Hallucination Detection in Large Language Models” on OpenResearch

About this paper

Varun Teja Chundru, Debasmita Biswas

A hallucination-detection pipeline combines DeBERTa-based natural-language inference with Monte Carlo Dropout, calibration, and lightweight ensembles to assess both response labels and prediction reliability. Fine-tuning on HaluEval reaches up to 0.932 accuracy and 0.931 F1 with MC Dropout, while DPO reduces detector-identified hallucinations in Qwen2.5-0.5B-Instruct from 85.5% to 37.7%; transfer to biomedical SciFact remains limited unless domain-matched pre-training is used.

Key insights

• MC Dropout mainly improves classification near the decision boundary: it raises accuracy and F1 to 0.932 and 0.931, while the best simple average ensemble reaches AUROC 0.979.
• Removing source context lowers overall F1 from about 0.91 to 0.82 and causes the largest drop in summarization, supporting the use of evidentiary context rather than response-only cues.
• General-domain transfer is unreliable for biomedical claims; PubMedBERT fine-tuned on SciFact substantially outperforms both zero-shot transfer and sequential adaptation.

Summary by alphaXiv.