Reproduce “VQA Therapy: Exploring Answer Differences by Visually Grounding Answers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chongyan Chen, Samreen Anjum, Danna Gurari
VQA Therapy introduces the VQA-AnswerTherapy dataset, which provides segmentation masks for all unique human answers to visual questions, and defines two new challenges for visually grounding answer diversity. The work finds that 15.7% of VQA questions have distinct visual groundings for their different answers, and current models struggle significantly with these multi-grounding scenarios.
Key insights
- A notable portion (15.7%) of visual questions in the VQA-AnswerTherapy dataset exhibit multiple answer groundings, where different valid answers refer to distinct visual evidence.
- Questions related to text recognition and location are more prevalent in multi-grounding scenarios, while color recognition questions often have single groundings.
- Existing state-of-the-art VQA and referring segmentation models perform poorly on the introduced challenges, particularly when multiple answer groundings are present, indicating a limitation in current model design.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.