Reproduce “Bit-Swap: Recursive Bits-Back Coding for Lossless Compression with Hierarchical Latent Variables” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Friso H. Kingma, Pieter Abbeel, Jonathan Ho
Bit-Swap introduces a recursive bits-back coding method that enables efficient lossless compression for deep hierarchical latent variable models. This approach significantly reduces the initial bit overhead of prior bits-back techniques, achieving superior compression rates such as 1.29 bits/dim on MNIST and 3.82 bits/dim on CIFAR-10, surpassing existing methods.
Key insights
- Recursively applying the bits-back principle across hierarchical latent variables substantially amortizes the initial bit overhead, making deep models feasible for practical compression.
- The immediate replenishment of the bitstream during the encoding/decoding process allows for efficient utilization of hierarchical latent variable models, which are known for better density estimation.
- Careful discretization of continuous latent variables, using a combination of uniform and equal-mass binning, is critical for achieving effective lossless compression with these models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.