Reproduce “Heterosynaptic Circuits Are Universal Gradient Machines” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Liu Ziyin, Isaac Chuang, Tomaso Poggio

This research from MIT and NTT Research proposes a theoretical framework demonstrating that heterosynaptic plasticity can serve as a universal mechanism for gradient-based meta-learning in biological circuits. It shows how efficient gradient computation can emerge robustly even with non-differentiable activations, explaining various forms of neuroplasticity and suggesting new designs for AI optimizers and hardware.

Key insights

  • Heterosynaptic plasticity can act as a primary mechanism for learning, enabling global gradient computation from local interactions, with Hebbian plasticity potentially emerging as a byproduct.
  • The proposed mechanism robustly implements gradient learning even with non-differentiable activation functions and random microscopic connectivity, implicitly smoothing the predictive model.
  • The concept suggests that specific brain anatomy might be less critical for gradient computation than the dynamic emergent properties of heterosynaptic circuits, offering new avenues for AI optimizer design and analog hardware.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.