Reproduce “FuGIBench: Evaluating Reliable Preparation in Streaming Video” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Changheng Cai, Zhichao Huang, Song Ren, Zhichen Zhao, Hongsen Liu, Chaoyang Zheng, Wenxi Wu, Yuan Fang and 3 more

FuGIBench evaluates whether video-based assistants can wait, prepare reversibly for a likely tool handover, respond when due, and withdraw after cancellation. Models choose among WAIT, PREPARE, and RESPONSE using causal video prefixes and a task contract. On evaluation, expanded video has shorter mean preparation lead but higher response alignment than native video plus LoRA, showing lead alone does not establish reliability.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.