Reproduce “Randomly initialized autoencoders: fixed points and edge-of-chaos” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Leonid Berlyand, Roman Sarapin, Yitzchak Shmalo, Victor Slavin, Sasha Sodin
The authors analyze stability in randomly initialized, fully connected autoencoders with centered Gaussian weights. They distinguish local stability, measured through Jacobian growth at a fixed input, from global stability, measured by uniform output shrinkage. For ReLU, local critical variance approaches 2 at large depth; with one narrower bottleneck, variance below 1 suffices for global attraction to zero, not useful reconstruction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.