Reproduce “FuGIBench: Evaluating Reliable Preparation in Streaming Video” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Changheng Cai, Zhichao Huang, Song Ren, Zhichen Zhao, Hongsen Liu, Chaoyang Zheng, Wenxi Wu, Yuan Fang and 3 more
FuGIBench evaluates whether video-based assistants can wait, prepare reversibly for a likely tool handover, respond when due, and withdraw after cancellation. Models choose among WAIT, PREPARE, and RESPONSE using causal video prefixes and a task contract. On evaluation, expanded video has shorter mean preparation lead but higher response alignment than native video plus LoRA, showing lead alone does not establish reliability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.