Reproduce “ABACUS: A Benchmark for Image Editing Models to Assess Object Category and Counting Under Scene Conservation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aymen Lassoued, Angelique Loesch, Romaric Audigier
ABACUS evaluates whether image edits deliver requested object counts while preserving existing instances and scene content. It compares source and edited images, matches objects one-to-one, and checks candidate additions with detectors, using a VLM only for rare ambiguous categories. On 600 images, its counts agree with human annotations on 0.82; the benchmark covers 80 COCO thing categories.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.