Reproduce “Generalized Variational Continual Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Noel Loo, Siddharth Swaroop, Richard E. Turner
GVCL-F combines GVCL and task-specific FiLM layers, retaining probabilistic shared weights while adapting shared features for known task IDs. On Easy-CHASY’s ten handwritten-symbol tasks, its final average accuracy (equal-weight mean test accuracy after all tasks) was 90.9 ± 0.3%, versus 82.6 ± 0.9% for HAT. This result is benchmark-specific: GVCL alone did not outperform Online EWC on larger benchmarks.
Key insights
- GVCL provides a theoretical unification, demonstrating that Online EWC is a limiting case of likelihood-tempered VCL as the tempering factor approaches zero.
- Task-specific FiLM layers effectively mitigate variational overpruning, allowing shared network parameters to revert to their prior distributions while preserving network capacity for new tasks.
- The Bayesian framework of GVCL-F offers increased robustness against overfitting in data-scarce continual learning scenarios and significantly improves uncertainty calibration.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.