Reproduce “Simple Cycle Reservoirs are Universal” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Boyu Li, Robert Simon Fong, Peter Tiňo
Simple Cycle Reservoirs (SCRs) and their variants are rigorously proven to be universal approximators of time-invariant fading memory filters in the complex domain. This establishes that these highly constrained recurrent neural network architectures, with internal dynamics governed by only a single tunable parameter, possess the same expressive power as more general linear reservoir systems.
Key insights
- Universality in Reservoir Computing does not necessitate complex or random internal connectivity; highly structured and simple cyclic architectures can achieve the same expressive power.
- Even with severe constraints on reservoir weights and input couplings (e.g., binary inputs), universal approximation can be achieved, often by increasing the reservoir size or using parallel, simpler structures.
- The crucial ingredients for universal temporal approximation might be far simpler than previously assumed, enabling more efficient and physically realizable hardware implementations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.