Reproduce “Balancing Continual Learning and Fine-tuning for Human Activity Recognition” on OpenResearch

About this paper

Chi Ian Tang, Lorena Qendro, Dimitris Spathis, Fahim Kawsar, Akhil Mathur, Cecilia Mascolo

This paper adapts and evaluates Continual Self-Supervised Learning (CSSL) frameworks for Human Activity Recognition (HAR) using wearable sensors, showing that Kaizen, a semi-supervised continual learning model, effectively mitigates catastrophic forgetting. The research from Nokia Bell Labs and the University of Cambridge introduces a progressive importance coefficient for knowledge distillation, achieving superior performance by dynamically balancing new task learning and knowledge retention.

Key insights

• Kaizen significantly outperforms baseline CSSL methods and fine-tuning without forgetting mitigation, achieving higher continual accuracy and substantially reduced catastrophic forgetting for HAR.
• The choice of Self-Supervised Learning (SSL) method impacts performance, with BYOL generally outperforming MoCoV2+ as the contrastive learning mechanism within the CSSL frameworks.
• A progressive importance coefficient (λ_C), which increases as more tasks are learned, is the most effective strategy for dynamically balancing stability and plasticity, leading to optimal overall continual learning performance.

Summary by alphaXiv.