Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SED: Self-Evaluation Decoding Enhances Large Language Models for Better Generation

Ziqin Luo, Haixia Han, Haokun Zhao, Guochao Jiang, Chengyu Du, Tingyun Li, Jiaqing Liang, Deqing Yang, Yanghua Xiao

TL;DR

Self-Evaluation Decoding (SED) targets chaotic points in autoregressive generation by introducing a three-stage process: chaotic-point detection, speculation over top tokens, and self-evaluation feedback that yields a propensity score for token selection. By combining evaluation-guided feedback with the original token probabilities, SED selects tokens that minimize error propagation at uncertain points. Training leverages multi-task learning from augmented evaluation and generation data, and a multi-model ensembling strategy strengthens evaluation signals. Across HotpotQA, XieZhi, and GSM8K with models like falcon-7b-instruct and llama2-7b-chat-hf, ratio-based SED consistently improves accuracy over greedy, beam, and nuclear decoding, underscoring the practical impact of incorporating self-evaluation into decoding. Alignment between speculation and generation is essential for gains, and higher-quality evaluation signals further boost performance.

Abstract

Existing Large Language Models (LLMs) generate text through unidirectional autoregressive decoding methods to respond to various user queries. These methods tend to consider token selection in a simple sequential manner, making it easy to fall into suboptimal options when encountering uncertain tokens, referred to as chaotic points in our work. Many chaotic points exist in texts generated by LLMs, and they often significantly affect the quality of subsequently generated tokens, which can interfere with LLMs' generation. This paper proposes Self-Evaluation Decoding, SED, a decoding method for enhancing model generation. Analogous to the human decision-making process, SED integrates speculation and evaluation steps into the decoding process, allowing LLMs to make more careful decisions and thus optimize token selection at chaotic points. Experimental results across various tasks using different LLMs demonstrate SED's effectiveness.

SED: Self-Evaluation Decoding Enhances Large Language Models for Better Generation

TL;DR

Self-Evaluation Decoding (SED) targets chaotic points in autoregressive generation by introducing a three-stage process: chaotic-point detection, speculation over top tokens, and self-evaluation feedback that yields a propensity score for token selection. By combining evaluation-guided feedback with the original token probabilities, SED selects tokens that minimize error propagation at uncertain points. Training leverages multi-task learning from augmented evaluation and generation data, and a multi-model ensembling strategy strengthens evaluation signals. Across HotpotQA, XieZhi, and GSM8K with models like falcon-7b-instruct and llama2-7b-chat-hf, ratio-based SED consistently improves accuracy over greedy, beam, and nuclear decoding, underscoring the practical impact of incorporating self-evaluation into decoding. Alignment between speculation and generation is essential for gains, and higher-quality evaluation signals further boost performance.

Abstract

Existing Large Language Models (LLMs) generate text through unidirectional autoregressive decoding methods to respond to various user queries. These methods tend to consider token selection in a simple sequential manner, making it easy to fall into suboptimal options when encountering uncertain tokens, referred to as chaotic points in our work. Many chaotic points exist in texts generated by LLMs, and they often significantly affect the quality of subsequently generated tokens, which can interfere with LLMs' generation. This paper proposes Self-Evaluation Decoding, SED, a decoding method for enhancing model generation. Analogous to the human decision-making process, SED integrates speculation and evaluation steps into the decoding process, allowing LLMs to make more careful decisions and thus optimize token selection at chaotic points. Experimental results across various tasks using different LLMs demonstrate SED's effectiveness.
Paper Structure (20 sections, 10 equations, 2 figures, 10 tables, 1 algorithm)

This paper contains 20 sections, 10 equations, 2 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Self-evaluation decoding
  4. Chaotic point detection
  5. Speculation and evaluation
  6. Feedback backpropagation
  7. Evaluation ability enhancing
  8. Experiment
  9. Datasets
  10. Baselines and metrics
  11. Experimental settings
  12. Main results
  13. Analysis of evaluation quality
  14. Analysis of generation-speculation alignment
  15. Conclusion
  16. ...and 5 more sections

Figures (2)

  • Figure 1: The same model with different decoding methods generates two answers to the GSM8K question. The left is generated through greedy search, while the right is produced via SED. Tokens with or colored are the first chaotic points the model meets. Underlines represent the final generated text after the chaotic point. Corresponding sentence-level probabilities and perplexities are shown next to them. The token probability mass at the chaotic point is shown in the left-bottom.
  • Figure 2: The workflow of token selection at ONE chaotic point during decoding. Other chaotic points also follow a similar process. In this example, we adopt the ratio-based detection criterion and consider top $3$ tokens for speculation and evaluation. We set $\alpha=0.8$ and $\delta_r=0.75$.