Reproduce “Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Training” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Margaret Li, Stephen Roller, Ilia Kulikov, Sean Welleck, Y-Lan Boureau, Kyunghyun Cho, Jason Weston

The authors train dialogue models to lower the probability of responses labeled as contradictions, adding an unlikelihood loss on contradiction candidates to likelihood training. On constructed two-utterance tests, preference for neutral candidates over contradictions rises from 18% to 78%; ConvAI2 perplexity changes from 11.4 to 11.9. These results measure selection on constructed tests, not consistency in unrestricted conversation.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.