Reproduce “Incremental Learning in Deep Convolutional Neural Networks Using Partial Network Sharing” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Syed Shakib Sarwar, Aayush Ankit, Kaushik Roy

An incremental learning methodology for deep convolutional neural networks (DCNNs) employs partial network sharing and a clone-and-branch technique to efficiently integrate new classes while preventing catastrophic forgetting and eliminating the need for previous training data. The method reduced computation energy by up to 2.45x and training time by up to 6x on benchmarks like CIFAR-100 and ImageNet, maintaining classification accuracy.

Key insights

  • An optimal trade-off exists between the extent of network parameter sharing and maintaining high classification accuracy for incrementally learned tasks.
  • Deeper DCNN architectures and robustly pre-trained base networks enable greater parameter sharing and yield more substantial energy and computational benefits.
  • Freezing shared layers in the 'clone-and-branch' architecture effectively mitigates catastrophic forgetting and eliminates the need to retain or re-access previous training data.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.