Reproduce “Learning infinite context windows in recurrent architectures via spatial neural computing” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Aleix Salvador-Pomarol, Arthur N. Montanari, Earl K. Miller, Adilson E. Motter, Jorge Cortés

SpatialRNN uses an evolving, wave-like medium to carry neuron activity forward, letting second-order dynamics represent earlier hidden states without a separate learned matrix for each delay. On a copy task, it recalls five binary tokens after zero-valued distractors. At sequence length 1024, it reached 99.04 ± 0.07% accuracy, versus 13.39 ± 3.07% for a finite-delay HORNN, a benchmark result rather than evidence of unlimited or lossless memory.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.