Reproduce “Learning to Jump: Thinning and Thickening Latent Counts for Generative Modeling” on OpenResearch

About this paper

Tianqi Chen, Mingyuan Zhou

The paper introduces "Learning to Jump," a new deep generative modeling framework and JUMP models, specifically designed for non-negative data with characteristics like sparsity, skewness, and overdispersion. JUMP models consistently outperform existing diffusion models on these challenging data types, offering a principled way to generate data from exact zeros through iterative count thickening.

Key insights

• The iterative count-thinning and thickening process, starting from exact zeros, allows JUMP models to naturally and precisely capture sparsity in data, a challenge for Gaussian-noise-based diffusion models.
• Employing a relative entropy-based loss (Bregman divergence) provides a theoretically sound optimization objective that is well-suited for the statistical characteristics of non-negative, often skewed or overdispersed, count data.
• "Learning to Jump" establishes a distinct paradigm for iterative generative modeling, demonstrating that physically intuitive transformations beyond mere denoising can lead to more robust and applicable DGM frameworks.

Summary by alphaXiv.