Reproduce “A Complete Characterization of Tensorizable $f$-divergences” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Rodrigo Cruz, Flavio P. Calmon, Qian Yu

This research provides a complete characterization of tensorizable f-divergences, a class of information-theoretic measures crucial for their compositional properties under product measures. The work formally demonstrates that only linear combinations of Kullback-Leibler and reverse Kullback-Leibler divergences, or power divergences (which include Hellinger divergences), possess this property, unifying their compositional rules into a single multi-affine family.

Key insights

  • Any admissible tensorization rule for f-divergences must follow a universal multi-affine form, simplifying the understanding of their compositional behavior.
  • A symmetrized tensorizable f-divergence is uniquely determined by its tensorization parameter (γ) and a 'local characteristic' (ℓ) quantifying its quadratic vanishing rate near identical distributions.
  • The set of convex functions 'f' inducing tensorizable f-divergences is restricted to those generating specific families: linear combinations of Kullback-Leibler and reverse Kullback-Leibler divergences, and power divergences.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.