Reproduce “Beyond Goldfish Memory: Long-Term Open-Domain Conversation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jing Xu, Arthur Szlam, Jason Weston

Researchers from Facebook AI Research introduced the Multi-Session Chat (MSC) dataset and novel summarization memory-augmented models to address the "goldfish memory" problem in open-domain conversational AI. Their SumMem-MSC models improved the ability to reference past user topics from 14.5% (BlenderBot baseline) to 33.8% in human evaluations, leading to more engaging multi-session dialogues.

Key insights

  • Existing state-of-the-art dialogue models exhibit "goldfish memory," performing poorly in multi-session contexts due to limited context windows.
  • Summarizing dialogue history provides a more effective and compact long-term memory representation than raw dialogue for retrieval-augmented models.
  • Retrieval-augmented models, especially those leveraging summarized memory, improve consistency and user engagement in long-term conversations by enabling reference to past interactions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.