Reproduce “DIVE: Benchmarking Full-Duplex Interaction with Visual Engagement in Continuous Streams” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuhang Chen, Guangyue Liu, Shilong Liu
DIVE evaluates whether real-time multimodal assistants respond appropriately, speak within valid time windows, and provide task-relevant content across six audio-visual tasks. It separates timing and participation decisions from response quality, rather than treating fast or correct answers as sufficient. Among tested native audio-visual models, GPT-Realtime-2.1 has the highest average window response rate, 53.77%, showing benchmark performance, not deployment outcomes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.