Reproduce “Continual Neural Topic Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Charu Karakkaparambil James, Waleed Mustafa, Marius Kloft, Sophie Fellenz
The Continual Neural Topic Model (CoNTM) learns topic models from sequential data streams while preserving long-term memory, demonstrating an average rank of 1.6 in topic quality and achieving lower predictive perplexity and faster runtime than baseline dynamic topic models on various datasets.
Key insights
- Combining local topic perturbations with a continuously updated global topic prior allows for tracking topic evolution and maintaining long-term memory in streaming data.
- The DVAE framework can be effectively adapted for continual learning in topic modeling by integrating explicit mechanisms for global knowledge retention and local adaptation.
- A controlled running average update strategy for global topic parameters provides robustness and stability to the continual learning process, ensuring balanced forgetting and adaptation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.