Reproduce “On the Cardinality of Optimal Representations in the Binary-Source Information Bottleneck” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dier Tang, Jun Chen
The authors prove that when the source alphabet is binary, an optimal information-bottleneck representation needs at most two output symbols, for any joint distribution with finite target and any rate constraint. They analyze posterior probabilities and use a curvature argument to reduce the representation’s alphabet. For source alphabets of size at least three, the universal two-symbol guarantee does not extend.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.