Reproduce “Generalized Denoising Auto-Encoders as Generative Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yoshua Bengio, Li Yao, Guillaume Alain, Pascal Vincent
Researchers at Universit
Key insights
- DAEs, when trained to accurately estimate the reverse conditional distribution P(X|˜X), can consistently define an alternating Markov chain whose stationary distribution converges to the true data-generating distribution P(X).
- The conditional distribution P(X|˜X) is often a simpler, more local distribution to approximate than the complex, multi-modal P(X) itself, making the DAE's learning task feasible.
- Walkback training effectively regularizes the generative model by explicitly pushing the DAE to learn to correctly navigate the generative path from corrupted states back to the data manifold, thus reducing spurious modes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.