Reproduce “Artificial Neuronal Ensembles with Learned Context Dependent Gating” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Matthew J. Tilley, Michelle Miller, David J. Freedman
LXDG learns input-dependent gates to select hidden units during sequential learning without an explicit task label. A gating network modulates hidden-layer activity, while sparsity and gate-recall losses shape the gates and EWC protects important weights. On permuted MNIST, LXDG+EWC reaches 96% mean accuracy across 50 tasks, versus 72 ± 1% for EWC alone. It uses earlier-task samples and is tested on two MNIST benchmarks.
Key insights
- Neural networks can be designed to autonomously infer task context from input data and dynamically allocate distinct subnetworks, mimicking biological neuronal ensembles.
- Combining a learned architectural mechanism (LXDG) with parameter regularization (EWC) provides a robust solution to the stability-plasticity dilemma in continual learning.
- The network's ability to learn flexible, context-dependent gates allows for both disentangled representation of distinct tasks and potential knowledge transfer between similar tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.