Reproduce “Kernel Memory Networks: A Unifying Framework for Memory Modeling” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Georgios Iatropoulos, Johanni Brea, Wulfram Gerstner

A theoretical framework called Kernel Memory Networks unifies diverse memory models, including Hopfield networks and Kanerva's SDM, by reinterpreting them through kernel methods and Support Vector Machines. This approach defines normative memory models that optimize noise robustness, yields a continuous memory model with exponential storage capacity, and explains the capacity scaling of existing architectures.

Key insights

  • Prominent memory models like Kanerva's SDM and Modern Hopfield Networks are special, often suboptimal, instances of the derived kernel memory networks.
  • The storage capacity of memory models can be directly explained by the Vapnik-Chervonenkis (VC) dimension of the kernel employed by individual neurons.
  • Kernel memory networks offer biological interpretations, mapping to non-linear dendritic processing and synaptic cross-talk in biological neurons.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.