Reproduce “Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Axel Laborieux, Friedemann Zenke
Researchers from Friedrich Miescher Institute for Biomedical Research developed Holomorphic Equilibrium Propagation (hEP), an extension of Equilibrium Propagation that computes exact loss gradients using finite-amplitude, complex-valued teaching signals through neural oscillations. This approach scales to deep convolutional neural networks on ImageNet 32x32, matching backpropagation's performance without requiring separate temporal phases and demonstrating robustness to noise.
Key insights
- Exact loss gradients can be computed using finite-amplitude teaching signals if the underlying network dynamics are holomorphic and the teaching signal follows a circular path in the complex plane.
- Gradient estimation can occur continuously ("online") without distinct temporal phases, as gradients are encoded in the first Fourier mode of neural oscillations, provided timescales are separated appropriately.
- Maintaining holomorphic dynamics throughout the network is crucial for hEP's effectiveness, enabling robust and accurate gradient computation even in the presence of noise.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.