Reproduce “STT-Arena: A More Realistic Environment for Tool-Using with Spatio-Temporal Dynamics” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tingfeng Hui, Hao Xu, Pengyu Zhu, Hongsheng Xin, Kun Zhan, Sen Su, Chunxiao Liu, Ning Miao

STT-Arena tests whether tool-using agents recover when changes invalidate ongoing plans. It evaluates 227 interactive tasks with temporal, spatial, and coupled conflicts, requiring agents to detect changes, replan, and verify outcomes or report infeasibility. Claude-4.6-Opus achieves 35.39% overall Pass@1, while STT-Agent-4B reaches 27.17%; these results are specific to the benchmark.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.