Reproduce “HABIT-Bench: Evaluating Habit Induction from Longitudinal Weak Evidence in Agent Memory” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junming Zhang, Rubin Wei, Xu Cheng, Hao Sun, Jiarui Wang, Ruoxi Cheng, Xiquan Wang, Peisong Wu and 2 more
HABIT-Bench tests whether assistants can infer behavioral defaults from user history and decide when those defaults apply, including exceptions, context, timing, and user endorsement. It evaluates four-choice decisions across 114 synthetic users, 37,152 sessions, and four domains. With GLM-5.3-Flash, A-MEM reaches 33.61% macro accuracy versus 33.21% for dense retrieval; compressed Full Memory reaches 41.98%, in this controlled benchmark.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.