Reproduce “GroundingSuite: Measuring Complex Multi-Granular Pixel Grounding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rui Hu, Lianghui Zhu, Yuxuan Zhang, Tianheng Cheng, Lei Liu, Heng Liu, Longjin Ran, Xiaoxin Chen and 2 more
GroundingSuite introduces an efficient automatic annotation framework, GSSculpt, a 9.56 million image-text pair training dataset, GSTrain-10M, and a multi-granular evaluation benchmark, GSEval, for pixel grounding. Models trained with GSTrain-10M achieve state-of-the-art performance, such as a cIoU of 68.9 on gRefCOCO, while GSEval reveals the limitations of existing models in fine-grained and multi-granular pixel grounding.
Key insights
- Combining advanced VLMs for spatial localization, context-aware text generation, and robust IoU-based noise filtering enables scalable creation of high-quality pixel grounding datasets.
- Current pixel grounding models and multimodal large language models (MLLMs) exhibit significant limitations in fine-grained part-level and multi-granular pixel grounding, even with strong general text understanding.
- New, diverse, and large-scale training data like GSTrain-10M is crucial for achieving superior generalization and state-of-the-art performance in complex pixel grounding tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.