Reproduce “EDVD-LLaMA: Explainable Deepfake Video Detection via Multimodal Large Language Model Reasoning” on OpenResearch

About this paper

Haoran Sun, Chen Cai, Huiping Zhuang, Kong Aik Lee, Lap-Pui Chau, Yi Wang

EDVD-LLaMA, a framework for explainable deepfake video detection developed by researchers from The Hong Kong Polytechnic University and other institutions, employs a multimodal large language model to provide accurate detection and verifiable reasoning. It integrates spatio-temporal feature extraction with fine-grained facial metrics to suppress MLLM hallucination, achieving superior generalization across diverse forgery methods and datasets.

Key insights

• Explicitly encoding temporal dependencies through grid layouts and dual-branch feature extraction effectively captures subtle spatio-temporal deepfake artifacts and global semantic inconsistencies in videos.
• Integrating structured, fine-grained facial dynamic metrics into an MLLM's Chain-of-Thought reasoning significantly reduces hallucination and enhances the verifiability and trustworthiness of explanations.
• A staged optimization strategy with rationale and classification losses, complemented by an evidence consistency regularizer, effectively trains the MLLM to provide both accurate detection and grounded explanations.

Summary by alphaXiv.