Reproduce “Towards a Comprehensive Theory of Reservoir Computing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Denis Kleyko, Christopher J. Kymn, E. Paxon Frady, Amy Loutfi, Friedrich T. Sommer
The authors use perceptron theory to predict echo state network recall accuracy and guide reservoir design. They estimate output-score statistics to assess whether a past symbol beats competing symbols, and introduce covariance-corrected readouts requiring no supervised training. In a tested linear recurrent-decay setting, fine-grained covariance readout performs best among untrained options; regression approaches it with sufficient data in that setting.
Key insights
- A generalized perceptron theory can accurately predict ESN recall accuracy and memory capacity across diverse architectures, provided the statistical moments of readout inner products are characterized.
- Optimal ESN memory capacity is achieved when recurrent decay is close to 1.0 and input scaling is close to 0.0, which encourages reservoir neurons to operate within their linear regime.
- Many ESN readout networks form a "simplex ETF" geometric structure, resembling "neural collapse" observed in deep neural networks, hinting at fundamental computational principles.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.