Reproduce “The Entropy of Floating-Point Numbers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sultan M. Daniels, Samuel H. D'Ambrosia, Michael R. DeWeese, Anant Sahai
The authors estimate Shannon entropy, in bits, for data rounded to an idealized floating-point format. They relate entropy to differential entropy, expected log bin volume, and a KL-divergence correction, then smooth changing bin widths with a function proportional to magnitude. In comparisons, the estimate tracks exact entropy across scale within the format’s usable range; this is approximate, not universal invariance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.