Reproduce “Stochastic Synapses Enable Efficient Brain-Inspired Learning Machines” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Emre O. Neftci, Bruno U. Pedroni, Siddharth Joshi, Maruan Al-Shedivat, Gert Cauwenberghs

The Synaptic Sampling Machine uses blank-out synapses that independently pass presynaptic events with probability 0.5, making omissions both sampling variability and connection-level regularization. Event-driven contrastive divergence combines these samples with local, phase-gated spike-timing-dependent plasticity. On MNIST, it reports 4.4% classification error versus 5.0% for an RBM, while operation-count comparisons do not establish lower hardware energy or faster training.

Key insights

  • Synaptic stochasticity serves a dual purpose, acting as an efficient mechanism for Monte Carlo sampling in probabilistic models and as a regularization technique similar to DropConnect.
  • S2Ms naturally learn sparse representations without explicit sparsity constraints, beneficial for reducing memory and power consumption in hardware.
  • The effective activation function of S2Ms, derived from synaptic unreliability, dynamically adjusts its slope based on network state, analogous to a thermal noise parameter.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.