Reproduce “EditJudge-Bench: Auditing VLM Image-Edit Judges with Synthetic Ground Truth” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jens Parslov, Marco Schouten, Dim P. Papadopoulos
The authors introduce EditJudge-Bench, which tests whether vision-language model judges verify instructed image edits. It pairs before-and-after indoor Blender renders from known scene interventions with factual and counterfactual instructions, covering 1,382 pairs across nine categories. GPT-4o with VIEScore reaches 69.0% AUROC, versus 71.5% for the strongest VLM; results concern controlled synthetic edits, not real-world edit quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.