Reproduce “Take A Step Back: Rethinking the Two Stages in Visual Reasoning” on OpenResearch

About this paper

Mingyu Zhang, Jiting Cai, Mingyu Liu, Yue Xu, Cewu Lu, Yong-Lu Li

Researchers from Shanghai Jiao Tong University propose a two-stage framework for visual reasoning, disentangling task-specific symbolization from a shared, generalizable reasoner. This approach allows a simple MLP reasoner to achieve performance comparable to state-of-the-art ad-hoc models across diverse visual reasoning benchmarks, demonstrating improved generalization on out-of-domain tasks when trained multi-domain.

Key insights

• Disentangling symbolization (perception) from logical reasoning leads to superior generalization capabilities in visual reasoning systems.
• The optimal depth and design for the symbolization stage vary significantly across different visual tasks, necessitating task-specific encoders.
• Surprisingly, a simple Multilayer Perceptron (MLP) can serve as an effective and efficient shared reasoner when provided with well-formed symbolic inputs.
• Training a shared reasoner on diverse, cross-domain datasets significantly enhances its ability to generalize to unseen, out-of-domain tasks.

Summary by alphaXiv.