Reproduce “Pro-Bench: Prompt-Robust Open-Vocabulary Visual Grounding Across Real-World Heterogeneous Environments” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Linus Nwankwo, Muslim Alaran, Christian Rauch, Stanley Chukwuebuka Obilikpa, Elmar Rueckert
Pro-Bench evaluates whether open-vocabulary models locate the same target when operators describe it differently in real-robot imagery. It tests 16 configurations zero-shot using original, categorical, and template-wrapped prompts. Ten configurations perform best with categorical prompts, five with templates, and only YOLOE-X with original queries. SAM3’s PCR is 0.024; its PLC is 0.984 only among targets recovered under every formulation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.