Reproduce “Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, Joshua B. Tenenbaum
A neural-symbolic Visual Question Answering (VQA) model, NS-VQA, separates visual perception and language parsing (using neural networks) from explicit reasoning (using a symbolic program executor). This approach achieves 99.8% accuracy on the CLEVR dataset while demonstrating high data and memory efficiency, along with robust generalization to novel attribute combinations, human-generated questions, and new scene contexts like Minecraft.
Key insights
- Disentangling perception and language understanding (neural) from core reasoning (symbolic) improves VQA accuracy, interpretability, and robustness to complex queries.
- Generating compact, structural scene representations from images drastically reduces memory requirements and facilitates efficient symbolic program execution.
- Training program parsers through a combination of limited supervised pretraining and reinforcement learning achieves high data efficiency for complex reasoning tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.