Reproduce “Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs” on OpenResearch

About this paper

Minh Nhat Nguyen, Andrew Baker, Clement Neo, Allen Roush, Andreas Kirsch, Ravid Shwartz-Ziv

The paper introduces min-p sampling, a dynamic truncation method that adjusts token selection based on a language model's confidence, preserving coherence while increasing output diversity. It achieves superior performance in reasoning, creative writing, and human evaluations, especially at higher generation temperatures.

Key insights

• The method's effectiveness stems from its *relative* truncation threshold, which automatically adjusts to the sharpness of the probability distribution, unlike static thresholds.
• Min-p specifically addresses the problem of coherence degradation at high temperatures by ensuring that only sufficiently probable tokens are considered, even when the overall distribution is flattened.
• It offers a simple, computationally lightweight, and efficient solution that integrates easily into existing LLM inference pipelines without significant overhead.

Summary by alphaXiv.