Reproduce “One Assistant, Many Memories: Benchmarking Real-World Group Assistance When Recall Is Not Enough” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhanghao Hu, Linhai Zhang, Qian Zhao, Qi Zhu, Yuan Hua, Di Liang, Jiasheng Si, Xin Zhao and 3 more
GroupAssistBench tests whether assistants both recall facts from group chats and use them in open-ended help. It distinguishes single memories, combined memories, and memories with updates or conflicts, using 2,395 questions from 50 Chinese-language conversations. EverMemOS scores 0.621 on recall versus 0.570 for matched full-context GPT-5.6-terra, but 0.247 on assistance versus 0.431, showing recall does not ensure effective assistance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.