Reproduce “Self-Contrastive Forward-Forward Algorithm” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xing Chen, Dongshu Liu, Jeremie Laydevant, Julie Grollier
The Self-Contrastive Forward-Forward (SCFF) algorithm, developed by researchers from Laboratoire Albert Fert, CNRS, Thales, and Université Paris-Saclay, enhances the original Forward-Forward (FF) algorithm's unsupervised learning capabilities for both images and sequential data. It achieves 80.75% accuracy on CIFAR-10 and 90.33% on a spoken digit dataset by employing a novel self-contrastive mechanism for positive and negative example generation, while preserving hardware-friendly properties.
Key insights
- A self-contrastive approach, creating positive examples from single instances and negative examples from pairs of distinct instances within a batch, provides an effective signal for FF's unsupervised learning.
- Reformulating inputs as sums (e.g., x_k + x_k) rather than concatenations simplifies computation while maintaining the contrastive objective due to shared weights and gradient convergence.
- SCFF's negative examples are statistically positioned between positive clusters in feature space, enhancing classification efficiency without requiring complex hard negative sampling.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.