Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Model-Based Minimum Bayes Risk Decoding for Text Generation

Yuu Jinnai, Tetsuro Morimura, Ukyo Honda, Kaito Ariu, Kenshi Abe

TL;DR

This work tackles decoding in text generation by addressing limitations of Minimum Bayes Risk (MBR) decoding that rely on Monte Carlo probability estimates. It introduces Model-Based MBMBR (MBMBR), which uses the model probability $P(y)$ directly over a restricted reference set to estimate the MBR objective, and proves that this approach minimizes KL divergence to the true distribution within the observed support. The authors provide analytical results and extensive experiments across machine translation, text summarization, image captioning, and data-to-text, showing MBMBR and its variant with length normalization often outperform standard MBR, with MBMBR requiring fewer samples to achieve similar divergence from the true distribution. The findings suggest MBMBR is a practical, broadly applicable decoding strategy for modern text generation systems, though performance can depend on the base model quality and task characteristics.

Abstract

Minimum Bayes Risk (MBR) decoding has been shown to be a powerful alternative to beam search decoding in a variety of text generation tasks. MBR decoding selects a hypothesis from a pool of hypotheses that has the least expected risk under a probability model according to a given utility function. Since it is impractical to compute the expected risk exactly over all possible hypotheses, two approximations are commonly used in MBR. First, it integrates over a sampled set of hypotheses rather than over all possible hypotheses. Second, it estimates the probability of each hypothesis using a Monte Carlo estimator. While the first approximation is necessary to make it computationally feasible, the second is not essential since we typically have access to the model probability at inference time. We propose Model-Based MBR (MBMBR), a variant of MBR that uses the model probability itself as the estimate of the probability distribution instead of the Monte Carlo estimate. We show analytically and empirically that the model-based estimate is more promising than the Monte Carlo estimate in text generation tasks. Our experiments show that MBMBR outperforms MBR in several text generation tasks, both with encoder-decoder models and with large language models.

Model-Based Minimum Bayes Risk Decoding for Text Generation

TL;DR

This work tackles decoding in text generation by addressing limitations of Minimum Bayes Risk (MBR) decoding that rely on Monte Carlo probability estimates. It introduces Model-Based MBMBR (MBMBR), which uses the model probability directly over a restricted reference set to estimate the MBR objective, and proves that this approach minimizes KL divergence to the true distribution within the observed support. The authors provide analytical results and extensive experiments across machine translation, text summarization, image captioning, and data-to-text, showing MBMBR and its variant with length normalization often outperform standard MBR, with MBMBR requiring fewer samples to achieve similar divergence from the true distribution. The findings suggest MBMBR is a practical, broadly applicable decoding strategy for modern text generation systems, though performance can depend on the base model quality and task characteristics.

Abstract

Minimum Bayes Risk (MBR) decoding has been shown to be a powerful alternative to beam search decoding in a variety of text generation tasks. MBR decoding selects a hypothesis from a pool of hypotheses that has the least expected risk under a probability model according to a given utility function. Since it is impractical to compute the expected risk exactly over all possible hypotheses, two approximations are commonly used in MBR. First, it integrates over a sampled set of hypotheses rather than over all possible hypotheses. Second, it estimates the probability of each hypothesis using a Monte Carlo estimator. While the first approximation is necessary to make it computationally feasible, the second is not essential since we typically have access to the model probability at inference time. We propose Model-Based MBR (MBMBR), a variant of MBR that uses the model probability itself as the estimate of the probability distribution instead of the Monte Carlo estimate. We show analytically and empirically that the model-based estimate is more promising than the Monte Carlo estimate in text generation tasks. Our experiments show that MBMBR outperforms MBR in several text generation tasks, both with encoder-decoder models and with large language models.
Paper Structure (36 sections, 3 theorems, 21 equations, 10 figures, 17 tables)

This paper contains 36 sections, 3 theorems, 21 equations, 10 figures, 17 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Minimum Bayes Risk Decoding
  4. Model-Based Minimum Bayes Risk (MBMBR) Decoding
  5. Mitigating the Length Bias.
  6. Properties of the Model-Based Estimate
  7. Probability Distribution of Model-Based Estimate
  8. Example.
  9. Divergence of the Model-Based Estimate to the True Distribution
  10. Analytical Results.
  11. Simulation Study of the Divergence of the Model-Based Distribution.
  12. Empirical Validation of Model-Based Estimates.
  13. Relationship of KL Divergence on Output Quality
  14. Analytical Results.
  15. Empirical Validation of the KL Divergence and the BLEU Score.
  16. ...and 21 more sections

Key Result

Theorem 4.1

The model-based distribution minimizes the Kullbuck-Leiber divergence over a collection of probability distributions with their support restricted to $\mathcal{H}_{\mathrm{ref}}$.

Figures (10)

  • Figure 1: (a) Kullback-Leibler divergence of the empirical distribution and the model-based distribution from the true model probability $P_\mathrm{{model}}$, averaged over the source sentences. (b) The correlation of the average KL divergence to the average BLEU score of the output. Evaluated on WMT'19 De-En.
  • Figure 2: Kullback-Leibler Divergence of the Monte Carlo estimate and the model-based estimate to the $P_\mathrm{{model}}$, averaged over the source sentences.
  • Figure 3: Jensen-Shannon Divergence of the Monte Carlo estimate and the model-based estimate to the $P_\mathrm{{model}}$, averaged over the source sentences.
  • Figure 4: BLEU score as a function of the number of samples on WMT'19 De-En and Ru-En (Section \ref{['sec:mtm']}).
  • Figure 5: BLEU score as a function of the number of samples on WMT'19 En-De and En-Ru (Section \ref{['sec:mtm']}).
  • ...and 5 more figures

Theorems & Definitions (3)

  • Theorem 4.1
  • Corollary 4.2
  • Lemma 4.3