Reproduce “Perplexity Cannot Always Tell Right from Wrong” on OpenResearch

About this paper

Petar Veličković, Federico Barbero, Christos Perivolaropoulos, Simon Osindero, Razvan Pascanu

The authors examine perplexity as a measure of model correctness. Under compact decoder-only Transformer and greedy-decoding assumptions, incorrect bitstring copies can approach correct copies in log-perplexity as sequences grow. In a parity experiment, micro-F1 and log-perplexity correlations were −0.94 on IID inputs and +0.31 on length-128 OOD inputs; the experiment does not establish a universal pattern or that confidence alone causes mismatch.

Key insights

• Perplexity can mislead evaluators by approaching arbitrarily low values even when a Transformer model is confidently making incorrect predictions, especially for sufficiently long sequences.
• The issues with perplexity stem from fundamental properties of the metric and model architecture, rather than solely from its aggregation method over long sequences.
• The interplay between model confidence and accuracy creates "unfavorable regions" in perplexity curves, where increased confidence can improve perplexity despite decreased accuracy, or vice-versa.

Summary by alphaXiv.