Reproduce “'Place-cell' emergence and learning of invariant data with restricted Boltzmann machines: breaking and dynamical restoration of continuous symmetries in the weight space” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Moshir Harsh, Jérôme Tubiana, Simona Cocco, Remi Monasson
Researchers from LPENS and Tel Aviv University demonstrated that Restricted Boltzmann Machines, trained on data exhibiting continuous invariances, autonomously develop localized "place-cell"-like receptive fields in their weight space. The work established that these fields emerge only above a critical data quantity and undergo long-term diffusion, and provided a theoretical explanation using reaction-diffusion dynamics.
Key insights
- RBMs trained on invariant data spontaneously form localized receptive fields whose width is proportional to the data's correlation length, demonstrating an emergent symmetry-breaking phenomenon in weight space.
- Individual receptive fields diffuse randomly over long training times, which effectively restores the global symmetry by allowing the network's entire map to traverse the invariant manifold, interpreted as a continuous attractor in weight space.
- A "retarded learning" transition occurs where a critical amount of training data is necessary for structured, localized receptive fields to emerge, linking learning capacity to data statistics and network size.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.