Reproduce “InViC: Intent-aware Visual Cues for Medical Visual Question Answering” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhisong Wang, Ziyang Chen, Zanting Ye, Hongze Zhu, Yefeng Zheng, Yong Xia
Researchers from institutions including Northwestern Polytechnical University and Westlake University developed InViC, a lightweight, plug-in framework that addresses "shortcut answering" in Medical Visual Question Answering (Med-VQA). InViC distills visual tokens into intent-aware cues and employs a two-stage bottleneck training strategy, resulting in consistent accuracy improvements across multiple MLLM backbones and Med-VQA benchmarks.
Key insights
- Architecturally constraining the flow of visual information through a bottlenecked pathway, specifically via intent-aware cue tokens, significantly enhances visual grounding and mitigates shortcut answering in Med-VQA models.
- The process of dynamically conditioning and refining cue token generation through question-aware initialization and cross-modal attention is crucial for effectively summarizing task-relevant visual evidence.
- A staged training approach, wherein a strict visual bottleneck is initially enforced (Stage I) and then relaxed (Stage II), effectively trains the model to rely on visual evidence while reducing train-test mismatch.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.