Reproduce “HEIE: MLLM-Based Hierarchical Explainable AIGC Image Implausibility Evaluator” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Fan Yang, Ru Zhen, Jianing Wang, Yanhao Zhang, Haoxiang Chen, Haonan Lu, Sicheng Zhao, Guiguang Ding
HEIE, developed by researchers from Tsinghua University and OPPO AI Center, introduces an MLLM-based framework to evaluate AIGC image implausibility, generating hierarchical heatmaps, overall plausibility scores, and detailed textual explanations. The system achieved state-of-the-art performance on standard benchmarks and surpassed commercial MLLMs in explanation quality, demonstrating strong generalization capabilities.
Key insights
- Combining MLLM reasoning with specialized visual mapping allows for both high-level semantic understanding and precise, fine-grained pixel-level defect localization.
- A Chain-of-Thought (CoT) prompting strategy significantly enhances MLLMs' ability to generate coherent textual explanations, accurate heatmaps, and plausible scores.
- An uncertainty-based adaptive fusion of global and local defect heatmaps improves accuracy by dynamically weighting contributions based on prediction confidence.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.