Reproduce “A Visual Grounding Benchmark for Retinal OCT with Multimodal Large Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ronald Fecso, José Morano, Botond Fazekas, Paul Widmann-Sedlnitzky, Dorota Himel, Laetita Hinterhuber, Virginia Mares, Andreas Pollreisz and 1 more

RetinaGroundBench evaluates whether multimodal language models localize queried biomarkers in retinal OCT images and answer presence questions. It pools 815 scans from two public datasets and scores boxes across 13 biomarker classes. The authors fine-tune the 8B RetGrounder, which reaches 33.9% macro F1, versus 23.7% for the best general-purpose model. Its recall is 27.5%, so most annotated findings remain missed.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.