Reproduce “Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shengyuan Ding, Xilin Wei, Xinyu Fang, Haodong Duan, Dahua Lin, Jiaqi Wang, Yuhang Zang
A new multimodal benchmark, RNG-Bench, is introduced to evaluate large language models' ability to reconstruct and act upon past observations in controllable Non-Markov games like Matching Pairs and 3D Maze. The study reveals that state-of-the-art MLLMs struggle with hidden-state tracking in these interactive environments, with performance significantly degrading as hidden information scales, but targeted fine-tuning can improve these capabilities.
Key insights
- MLLMs exhibit a substantial "Memory Gap" in Non-Markov games, indicating significant difficulty in maintaining and utilizing internal belief states for action, with performance dropping sharply as the scale of hidden information increases.
- Visual recognition limitations, the necessity of explicit textual action history for belief-state tracking, and challenges in translating internal spatial maps into effective planning are key bottlenecks for current MLLMs.
- Fine-tuning MLLMs on optimal-policy rollouts and filtered model demonstrations can effectively enhance targeted "remember-to-act" capabilities and generalize to unseen, larger problem instances without significantly degrading general multimodal abilities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.