Reproduce “Reservoir Computing meets Recurrent Kernels and Structured Transforms” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jonathan Dong, Ruben Ohana, Mushegh Rafayelyan, Florent Krzakala
A rigorous proof of convergence for Reservoir Computing to a deterministic recurrent kernel in the infinite-width limit, establishing an O(1/√N) rate, is presented. Additionally, Structured Reservoir Computing (SRC) is introduced, reducing the computational complexity of the recurrent step from O(N^2) to O(N log N) and demonstrating comparable performance on chaotic time series prediction tasks at significantly larger reservoir sizes.
Key insights
- Reservoir Computing states converge to a deterministic Recurrent Kernel in the infinite-width limit, providing a strong theoretical foundation for RC's empirical success.
- Structured transforms can be effectively integrated into Reservoir Computing (SRC) to drastically improve computational efficiency (O(N log N) vs O(N^2)) without significant performance loss.
- Practical convergence of RC to its kernel limit occurs even when some theoretical assumptions, such as weight resampling at each time step, are relaxed.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.