Reproduce “Closed-Loop Data Transcription to an LDR via Minimaxing Rate Reduction” on OpenResearch

About this paper

Xili Dai, Shengbang Tong, Mingyang Li, Ziyang Wu, Michael Psenka, Kwan Ho Ryan Chan, Pengyuan Zhai, Yaodong Yu and 3 more

A framework learns structured generative models by establishing a closed-loop transcription between data and a Linear Discriminative Representation (LDR), using a minimax game over an information-theoretic rate reduction metric. This approach yields high-fidelity data generation, explicit latent spaces with disentangled attributes, and effective classification capabilities, outperforming several baselines in FID scores.

Key insights

• The rate reduction metric provides a computable and robust measure for distances between high-dimensional, low-support data distributions, overcoming intractability issues of other metrics.
• The closed-loop feedback mechanism intrinsically unifies auto-encoding and adversarial learning within a single minimax objective, allowing for stable training without separate discriminators or complex regularization techniques.
• The framework explicitly learns an LDR where different classes map to orthogonal principal subspaces and visual attributes are disentangled along principal components, offering interpretable and controllable latent spaces.

Summary by alphaXiv.