Reproduce “Rewarding Chatbots for Real-World Engagement with Millions of Users” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Robert Irvine, Douglas Boubert, Vyas Raina, Adian Liusie, Ziyi Zhu, Vineet Mudupalli, Aliaksei Korshuk, Zongyi Liu and 6 more
Chai Research developed a scalable method for training engaging social chatbots by leveraging automatic pseudo-labels derived from millions of real-world user interactions. This approach resulted in over 30% increase in Day 30 user retention and up to 50% increase in mean conversation length.
Key insights
- Implicit human feedback, such as conversation continuation and avoidance of retries, provides a scalable and effective signal for training highly engaging chatbots.
- Scaling the training data for the reward model significantly improves performance, with a linear relationship observed between the logarithm of training data size and percentage improvement in Mean Conversation Length (MCL).
- Optimizing for the intersection of 'conversation continues' AND 'user does not retry the message' as a pseudo-label yields the best overall performance, particularly for long-term user retention.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.