Reproduce “Distributional Metrics for Evaluating Spoken Conversational Systems” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shree Harsha Bokkahalli Satish, Erica Cooper, Patrícia Schmidtová, Maike Züfle, Éva Székely, Nicholas Sanders, Ondřej Klejch

The Conversational Distribution Score (CDS) evaluates dialogue by comparing distributions of pace, rhythm, and turn-interaction behaviours in conversations with two human or synthetic reference distributions. On VoiceArena Goals, its composite matched listeners’ rankings on five of six system pairs with HSC–MSC, versus three of six with HSC–Behavior-SD. This out-of-domain result complements, rather than replaces, interaction tests and listening judgments.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.