Reproduce “MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detection” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chenqi Kong, Anwei Luo, Peijun Bao, Yi Yu, Haoliang Li, Zengwei Zheng, Shiqi Wang, Alex C. Kot
MoE-FFD introduces a framework for face forgery detection that integrates Mixture-of-Experts with Parameter-Efficient Fine-Tuning on a frozen Vision Transformer backbone. The method attains state-of-the-art generalization and robustness against unseen forgeries and real-world perturbations, achieving an average frame-level AUC of 86.78% on unseen datasets while utilizing minimal activated parameters.
Key insights
- Combining parameter-efficient modules (LoRA for global context and Convpass Adapters for local priors) with a frozen ViT backbone effectively balances performance and computational efficiency for FFD.
- Dynamically selecting specialized experts via a Mixture-of-Experts architecture significantly enhances model generalizability by adapting to diverse forgery characteristics without increasing inference cost.
- Explicitly modeling different types of local anomalies using specialized convolutional experts within the MoE framework improves the detection of subtle, localized forgery cues.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.