Reproduce “ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kwan Ho Ryan Chan, Yaodong Yu, Chong You, Haozhi Qi, John Wright, Yi Ma
This research introduces ReduNet, a "white-box" deep network architecture and training framework derived from the principle of maximizing rate reduction. The framework enables the principled construction of deep convolutional networks whose layers and operators have explicit optimization, statistical, and geometric interpretations, leading to the learning of robust, discriminative, and invariant representations.
Key insights
- Deep network architectures and operators can be systematically derived from a principled objective (MCR^2), offering a "white-box" interpretation of their function rather than relying on empirical design.
- The MCR^2 principle inherently promotes both within-class feature compressibility and between-class feature discriminativeness, leading to representations that capture intrinsic data structures.
- Core deep learning concepts like layered architectures, skip connections, parallel processing groups, and multi-channel convolutions emerge naturally from the optimization of MCR^2 under specific invariance constraints.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.