Reproduce “Input complexity and out-of-distribution detection with likelihood-based generative models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Joan Serrà, David Álvarez, Vicenç Gómez, Olga Slizovskaia, José F. Núñez, Jordi Luque

Researchers from Dolby Laboratories, Telefónica Research, and Universitat Politècnica de Catalunya/Universitat Pompeu Fabra identified input complexity as the primary reason deep generative models often assign high likelihoods to out-of-distribution data. They developed a parameter-free OOD score, S(x), by subtracting input complexity from negative log-likelihoods, which improved AUROC scores for OOD detection from below 0.1 to near 1.0 on challenging datasets like MNIST and SVHN, outperforming many state-of-the-art methods.

Key insights

  • Generative model likelihoods are excessively influenced by input complexity, exhibiting strong negative correlation.
  • The "puzzling result" of high OOD likelihoods is primarily due to generative models assigning higher likelihoods to simpler inputs, regardless of distribution.
  • Subtracting a complexity estimate from log-likelihoods effectively corrects this bias, making likelihoods reliable for OOD detection.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.