Reproduce “IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jinzhao Li, Yinuo Chen, Wenxuan Song, Yijia Lei, Yichi Zhang, Honglei Yan, Panwang Pan, Miao Liu
IPIBench introduces a new benchmark for evaluating Multimodal Large Language Models (MLLMs) on interactive proactive intelligence within continuous video streams, uncovering limitations in existing models and demonstrating significant improvements with the proposed training-free IPI-Agent framework.
Key insights
- Current MLLMs, even advanced proprietary ones, demonstrate unstable proactive triggering (premature or delayed) and weak multi-turn interaction coordination, falling significantly short of human performance.
- Explicit architectural components for interaction control (managing reactive, proactive, and management instructions) and temporal gating are crucial for robust interactive proactive intelligence in MLLMs.
- Pure embedding-based semantic similarity is insufficient for reliable proactive triggering, highlighting the need for higher-level reasoning and contextual awareness in streaming MLLMs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.