Reproduce “Modeling Real-Time Interactive Conversations as Timed Diarized Transcripts” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Garrett Tanzer, Gustaf Ahdritz, Luke Melas-Kyriazi
This research introduces a method to simulate real-time interactive conversations using pretrained text-only language models by representing dialogues as timed diarized transcripts. The approach, employing causal rejection sampling, enables models to handle interruptions and generate fluid responses that mimic human conversational dynamics across instant messaging and spoken interactions.
Key insights
- Pretrained text-only language models can be adapted to model complex temporal and speaker dynamics of real-time human conversation by formatting input as timed diarized transcripts.
- A decoding strategy combining causal rejection sampling with a reaction time window and adapted speculative decoding allows LMs to maintain real-time interactivity while handling interruptions.
- The approach is generalizable across different conversational modalities, such as instant messaging and spoken dialogues, and can run on commodity hardware with minimal specialized data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.