Reproduce “IE-Bench: Advancing the Measurement of Text-Driven Image Editing for Human Perception Alignment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shangkun Sun, Bowen Qu, Xiaoyu Liang, Songlin Fan, Wei Gao
Researchers from Peking University and PengCheng Laboratory introduce IE-Bench, a comprehensive benchmark for evaluating text-driven image editing, comprising IE-DB, the first human-annotated dataset, and IE-QA, a multi-modal evaluation model. IE-QA achieves significantly higher correlation with human judgment (e.g., SROCC of 0.7520) compared to existing metrics, providing a standardized tool for assessing editing quality.
Key insights
- Image editing evaluation fundamentally differs from image generation assessment, requiring explicit consideration of the source image and its relationship to the edited output.
- A robust evaluation metric for image editing must simultaneously account for text-prompt consistency, preservation of relevant source content (source-target fidelity), and overall visual quality.
- Human subjective evaluation is critical for accurately assessing complex image editing tasks, necessitating a large-scale, carefully curated dataset with human annotations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.