Reproduce “Deep Fusion Memory: A Chunk-Level Parametric Memory for Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rubin Wei, Siyuan Sun, Hao Sun, Yuliang Liu, Xiang Shi, Jiarui Wang, Junming Zhang, Zhouhan Lin
DFM predicts one shared memory for each text chunk and injects it into decoder hidden layers for question answering without online retrieval. It trains on retrieval-derived memory targets while keeping the backbone fixed. Across six QA tasks, its unweighted average is 27.21 versus 24.14 for the strongest baseline on Mistral-7B-v0.3; it does not lead every task, including 2Wiki.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.