Reproduce “Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiayi Kuang, Jingyou Xie, Haohao Luo, Ronghao Li, Zhe Xu, Xianfeng Cheng, Yinghui Li, Xika Lin and 1 more
This survey provides a comprehensive and up-to-date overview of Visual Question Answering (VQA), focusing on the transformative impact of Multimodal Large Language Models (MLLMs) and advanced knowledge reasoning techniques on natural language understanding and inference in visual contexts.
Key insights
- MLLMs have become central to VQA, enabling zero-shot capabilities and sophisticated reasoning by integrating visual perception directly or through LLM-aided visual understanding strategies.
- VQA systems have evolved from simple feature extraction and fusion to advanced methods like Vision-Language Pre-training (VLP) models, which align and fuse visual and textual information using Transformer architectures.
- Knowledge reasoning is increasingly vital, moving beyond superficial understanding to incorporate external structured and unstructured knowledge, facilitated by multi-hop inference and MLLM-driven reasoning paradigms.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.