Reproduce “InEdit-Bench: Benchmarking Intermediate Logical Pathways for Intelligent Image Editing Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhiqiang Sheng, Xumeng Han, Zhiwei Zhang, Zenghui Xiong, Yifan Ding, Aoxiang Ping, Xiang Li, Tong Guo and 1 more
InEdit-Bench introduces a novel benchmark and evaluation framework designed to measure intelligent image editing models' capacity for generating coherent intermediate logical pathways during multi-step transformations. Evaluations of 14 leading models demonstrate significant limitations in dynamic reasoning and multi-stage causal understanding, with even the top performer achieving an accuracy of only 16.75% on perfect samples.
Key insights
- Current intelligent image editing models, including state-of-the-art proprietary solutions, exhibit widespread and significant shortcomings in multi-step editing and dynamic reasoning capabilities.
- Model performance generally declines as task complexity increases, particularly when moving from continuous transformations to discrete changes and tasks requiring adherence to scientific principles.
- The 'LMM-as-a-Judge' methodology proved highly reliable for complex visual task evaluation, demonstrating a Pearson correlation coefficient of 0.96 with human assessments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.