Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

M-Ped: Multi-Prompt Ensemble Decoding for Large Language Models

Jiaxin Guo, Daimeng Wei, Yuanchang Luo, Shimin Tao, Hengchao Shang, Zongyao Li, Shaojun Li, Jinlong Yang, Zhanglin Wu, Zhiqiang Rao, Hao Yang

TL;DR

This work introduces M-Ped, a multi-prompt ensemble decoding framework that improves LLM generation by averaging token distributions across multiple prompts processed in a single batch. It relies on Inner-Batch Ensemble to aggregate probabilities at each decoding step and Left-Padding to enable uniform batch processing with a single model, avoiding additional training. Across machine translation, code generation, and text simplification, M-Ped yields consistent gains in d-BLEU, pass@k, and LENS metrics, with 2–3 prompts often sufficing and larger models sometimes offering bigger improvements. The method remains model-agnostic and practical, and it can synergize with MBR for further gains, suggesting broad applicability for enhancing generation quality in diverse NLP tasks.

Abstract

With the widespread application of Large Language Models (LLMs) in the field of Natural Language Processing (NLP), enhancing their performance has become a research hotspot. This paper presents a novel multi-prompt ensemble decoding approach designed to bolster the generation quality of LLMs by leveraging the aggregation of outcomes from multiple prompts. Given a unique input $X$, we submit $n$ variations of prompts with $X$ to LLMs in batch mode to decode and derive probability distributions. For each token prediction, we calculate the ensemble probability by averaging the $n$ probability distributions within the batch, utilizing this aggregated probability to generate the token. This technique is dubbed Inner-Batch Ensemble. To facilitate efficient batch inference, we implement a Left-Padding strategy to maintain uniform input lengths across the n prompts. Through extensive experimentation on diverse NLP tasks, including machine translation, code generation, and text simplification, we demonstrate the efficacy of our method in enhancing LLM performance. The results show substantial improvements in BLEU scores, pass@$k$ rates, and LENS metrics over conventional methods.

M-Ped: Multi-Prompt Ensemble Decoding for Large Language Models

TL;DR

This work introduces M-Ped, a multi-prompt ensemble decoding framework that improves LLM generation by averaging token distributions across multiple prompts processed in a single batch. It relies on Inner-Batch Ensemble to aggregate probabilities at each decoding step and Left-Padding to enable uniform batch processing with a single model, avoiding additional training. Across machine translation, code generation, and text simplification, M-Ped yields consistent gains in d-BLEU, pass@k, and LENS metrics, with 2–3 prompts often sufficing and larger models sometimes offering bigger improvements. The method remains model-agnostic and practical, and it can synergize with MBR for further gains, suggesting broad applicability for enhancing generation quality in diverse NLP tasks.

Abstract

With the widespread application of Large Language Models (LLMs) in the field of Natural Language Processing (NLP), enhancing their performance has become a research hotspot. This paper presents a novel multi-prompt ensemble decoding approach designed to bolster the generation quality of LLMs by leveraging the aggregation of outcomes from multiple prompts. Given a unique input , we submit variations of prompts with to LLMs in batch mode to decode and derive probability distributions. For each token prediction, we calculate the ensemble probability by averaging the probability distributions within the batch, utilizing this aggregated probability to generate the token. This technique is dubbed Inner-Batch Ensemble. To facilitate efficient batch inference, we implement a Left-Padding strategy to maintain uniform input lengths across the n prompts. Through extensive experimentation on diverse NLP tasks, including machine translation, code generation, and text simplification, we demonstrate the efficacy of our method in enhancing LLM performance. The results show substantial improvements in BLEU scores, pass@ rates, and LENS metrics over conventional methods.

Paper Structure

This paper contains 44 sections, 3 equations, 2 figures, 21 tables.

Table of Contents

  1. Introduction
  2. Approach
  3. Inner-Batch Ensemble
  4. Left-Padding
  5. Experiments
  6. Application to Machine Translation Task
  7. Datasets
  8. Model
  9. Evaluate Metrics
  10. Prompts Design
  11. Results
  12. Application to Code Generation Task
  13. Dataset
  14. Model
  15. Evaluate Metrics
  16. ...and 29 more sections

Figures (2)

  • Figure 1: (a) Standard Ensemble: Input $X$ with prompt $P$ is fed into models $M_1$, $M_2$, ..., $M_n$, yielding predictions $Y_1$, $Y_2$, ..., $Y_n$. These are combined to get the final ensemble result $Y$. (b) Our Multi-Prompt Ensemble: Input $X$ constructs prompts $P_1$, $P_2$, ..., $P_n$, forming diverse samples batched through one model $M$ to get predictions $Y_1$, $Y_2$, ..., $Y_n$. The ensemble result $Y$ is the average of these predictions.
  • Figure 2: The overall process of Our Multi-Prompt Ensemble Decoding.