Reproduce “Lifelong Generative Modeling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jason Ramapuram, Magda Gregorova, Alexandros Kalousis
A Lifelong Generative Model (LGM) is introduced by researchers at the University of Geneva and HES-SO Geneva for unsupervised continual learning, effectively mitigating catastrophic forgetting and reducing sample complexity by leveraging a student-teacher VAE architecture without retaining past data or models.
Key insights
- It is possible to continuously learn and generate from sequential data distributions without storing prior data or model parameters, leveraging a single evolving model.
- A combination of generative replay and aligning latent posteriors (functional regularization) between student and teacher models is effective for knowledge transfer and mitigating catastrophic forgetting.
- Decomposing the latent space into discrete and continuous components, coupled with mutual information regularization, facilitates disentangled representation learning and controlled generation across different learned distributions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.