Reproduce “Predictive Attractor Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ramy Mounir, Sudeep Sarkar

Researchers at the University of South Florida developed Predictive Attractor Models (PAM), a novel sequential memory architecture combining predictive coding and attractor networks, which effectively mitigates catastrophic forgetting, uniquely encodes context, and generates distinct future possibilities from ambiguous inputs. PAM demonstrated superior capacity for long, correlated sequences and high robustness to noisy data, operating efficiently on standard CPUs.

Key insights

  • Integrating attractor networks (Hopfield-like) with predictive coding allows for representing multiple distinct future possibilities from a given context.
  • Employing a context dimension within Sparse Distributed Representations (SDRs) enables unique encoding of high-order sequential context, preventing catastrophic forgetting and improving capacity.
  • Local Hebbian learning with a near-zero forgetting coefficient in the transition matrix facilitates continuous learning of a union of possibilities, not just averages, which is crucial for avoiding catastrophic forgetting and supporting stochastic generation.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.