Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs

Minh Nhat Nguyen, Andrew Baker, Clement Neo, Allen Roush, Andreas Kirsch, Ravid Shwartz-Ziv

TL;DR

<3-5 sentence high-level summary>

Abstract

Large Language Models (LLMs) generate text by sampling the next token from a probability distribution over the vocabulary at each decoding step. Popular sampling methods like top-p (nucleus sampling) often struggle to balance quality and diversity, especially at higher temperatures which lead to incoherent or repetitive outputs. We propose min-p sampling, a dynamic truncation method that adjusts the sampling threshold based on the model's confidence by using the top token's probability as a scaling factor. Our experiments on benchmarks including GPQA, GSM8K, and AlpacaEval Creative Writing show that min-p sampling improves both the quality and diversity of generated text across different model families (Mistral and Llama 3) and model sizes (1B to 123B parameters), especially at higher temperatures. Human evaluations further show a clear preference for min-p sampling, in both text quality and creativity. Min-p sampling has been adopted by popular open-source LLM frameworks, including Hugging Face Transformers, VLLM, and many others, highlighting its considerable impact on improving text generation quality.

Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs

TL;DR

<3-5 sentence high-level summary>

Abstract

Large Language Models (LLMs) generate text by sampling the next token from a probability distribution over the vocabulary at each decoding step. Popular sampling methods like top-p (nucleus sampling) often struggle to balance quality and diversity, especially at higher temperatures which lead to incoherent or repetitive outputs. We propose min-p sampling, a dynamic truncation method that adjusts the sampling threshold based on the model's confidence by using the top token's probability as a scaling factor. Our experiments on benchmarks including GPQA, GSM8K, and AlpacaEval Creative Writing show that min-p sampling improves both the quality and diversity of generated text across different model families (Mistral and Llama 3) and model sizes (1B to 123B parameters), especially at higher temperatures. Human evaluations further show a clear preference for min-p sampling, in both text quality and creativity. Min-p sampling has been adopted by popular open-source LLM frameworks, including Hugging Face Transformers, VLLM, and many others, highlighting its considerable impact on improving text generation quality.
Paper Structure (110 sections, 2 equations, 2 figures, 18 tables)

This paper contains 110 sections, 2 equations, 2 figures, 18 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Greedy Decoding and Beam Search.
  4. Stochastic Sampling Methods.
  5. Entropy-Based Sampling.
  6. Dynamic Threshold Methods.
  7. Min-p Sampling
  8. Overview of Min-p Sampling
  9. Intuition Behind Min-p Sampling
  10. Advantages Over Existing Methods
  11. Balancing Creativity and Coherence.
  12. Robustness at High Temperatures.
  13. Simplicity.
  14. Implementation Details
  15. Integration into Decoding Pipelines.
  16. ...and 95 more sections

Figures (2)

  • Figure 1: Comparison of sampling methods. (a) Top-$p$ sampling in a high-certainty case. (b) Top-$k$ sampling in a low-certainty case. (c) Min-$p$ sampling in a high-certainty case. (d) Min-$p$ sampling in a low-certainty case. Min-$p$ sampling dynamically adjusts its threshold based on the model's confidence, focusing on likeliest tokens when confident and allowing more diverse options when uncertain. This dynamic threshold balances coherence and diversity better than top-$p$ and top-$k$.
  • Figure 2: Comparing min-$p$ and top-$p$ on GSM8K CoT-SC: Accuracy vs. Diversity (Mistral-7B). Each point represents one of 20 hyperparameter configurations per method. Accuracy-diversity tradeoff—measured by average entropy of correct predictions—on the GSM8K CoT-SC task shows that min-$p$(circles) achieves higher accuracy and diversity compared to top-$p$(triangles). Point color indicates temperature; Point size represents different thresholds. Solid lines show Pareto frontiers for each sampling method. The inset plot highlights min-$p$'s broader parameter coverage.