Reproduce “DynEval: Holistic Evaluations of T2I Generative Models in the Wild” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shyam Marjit, Dheeraj Baiju, Anuj Shikarkhane, Akhil Sakthieswaran, Sayak Paul, Anirban Chakraborty
DynEval evaluates text-to-image outputs by generating prompt-specific questions about requested content and visible distortions, then combining these with image-grounded quality checks derived from a scene graph. Its scores track human rankings on benchmarks; DynEval-4B reaches 0.8022 SRCC on TIFA, versus 0.7604 for FGA-BLIP2-OS. It ranks first on 9 of 11 benchmarks, but trails larger evaluators on RichHF and GenEval.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.