Reproduce “Memory Replay with Data Compression for Continual Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Liyuan Wang, Xingxing Zhang, Kuo Yang, Longhui Yu, Chongxuan Li, Lanqing Hong, Shifeng Zhang, Zhenguo Li and 2 more
Memory Replay with Data Compression (MRDC) enables continual learning models to store a greater quantity of past data within a fixed memory buffer, leveraging data compression. This approach consistently improved averaged incremental accuracy on benchmarks like ImageNet-sub and CUB-200-2011, and was shown to significantly alleviate forgetting, while an efficient Determinantal Point Process-based method dynamically selected optimal compression quality with minimal computational overhead.
Key insights
- Storing a higher quantity of compressed samples effectively mitigates catastrophic forgetting, outperforming strategies that store fewer uncompressed samples for the same memory footprint.
- Determinantal Point Processes offer an efficient, non-empirical approach to analytically determine the best compression quality by assessing the balance between data quantity and feature representation consistency.
- MRDC provides comparable or superior performance gains to other advanced continual learning baselines with significantly lower additional computational training costs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.