Reproduce “Heard Enough to Act: Early Tool Calling in Spoken Dialogue Systems” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yizhou Peng, Chen Chen, Donghang Wu, Hexin Liu, Eng Siong Chng
The authors train a spoken agent to decide after each 0.6-second audio chunk whether speech grounds a tool and all required arguments, emitting a partial transcript or tool-call payload. Labels mark the earliest actionable point, though annotation gates can delay it. Correct in-domain calls average an estimated 2.01-second lead over a turn-end baseline; about 59% of early fires are spurious.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.