Reproduce “PhiNets: Brain-inspired Non-contrastive Learning Based on Temporal Prediction Hypothesis” on OpenResearch

About this paper

Satoki Ishikawa, Makoto Yamada, Han Bao, Yuki Takezawa

Researchers from Science Tokyo, OIST, and Kyoto University developed PhiNet, a brain-inspired non-contrastive self-supervised learning model, to computationally validate the temporal prediction hypothesis of hippocampal function. PhiNet and its extension, X-PhiNet, demonstrate enhanced robustness against representational collapse, especially under varied hyperparameters, and achieve superior adaptivity in online and continual learning benchmarks by incorporating mechanisms for fast and slow learning.

Key insights

• The "temporal prediction hypothesis" from neuroscience can be successfully translated into a computationally effective machine learning model, leading to improved learning dynamics and stability.
• PhiNet's additional CA1 predictor ('g') and dual loss structure provide a wider retraction basin in its learning dynamics, making it inherently more robust against representational collapse than SimSiam, especially under various hyperparameter settings.
• Implementing Complementary Learning Systems (CLS) theory via an EMA-based long-term memory (X-PhiNet) significantly enhances adaptivity and reduces forgetting in online and continual learning scenarios, demonstrating the practical benefits of brain-inspired architectures.

Summary by alphaXiv.