Reproduce “Dense Associative Memory with Epanechnikov Energy” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Benjamin Hoover, Zhaoyang Shi, Krishnakumar Balasubramanian, Dmitry Krotov, Parikshit Ram
Dense Associative Memory with Epanechnikov Energy introduces a compact-support log-sum-ReLU energy that can preserve stored patterns as exact attractors while also creating additional minima at centroids of overlapping memories. Theory and experiments show exponential-in-dimension storage under separation conditions, increased diversity over log-sum-exponential energies, and plausible interpolations in learned image-latent spaces, with the tradeoff that retrieval fails outside the support of the stored patterns.
Key insights
- Compact support permits some memories to interact and form new attractors while leaving isolated memories unchanged, avoiding the exact-recall/emergence tradeoff identified for finite-temperature log-sum-exponential energies.
- Emergent minima have a simple geometric interpretation: they are centroids of subsets of stored patterns whose support balls overlap, and their number can greatly exceed the number of explicitly stored memories.
- The mechanism is representation-dependent: latent spaces can turn centroid minima into plausible image interpolations, whereas pixel-space averaging tends to produce less meaningful results.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.