Reproduce “Same Words, Different Actions: Paired Turn-Taking Evaluation under Rewritten Dialogue Contexts” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shuofeng Zhao, Hongwei Cai, Wenke Fan, Qingxiang Guo, Zhou Wang, Dawei Yang, Zhiyang Zhou, Yingxin Shang and 4 more
The authors introduce ECHO, a synthetic benchmark that tests whether systems YIELD to interruptions but KEEP speaking on backchannels. It pairs cases with the same insertion transcript and different preceding dialogue, then measures whether both actions are correct. On 100 interruption–backchannel pairs, Easy Turn scores 0.00% PASR, while MiniCPM-o 4.5 scores 54.00%; these event-level results do not establish natural conversation performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.