Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Predicting Compact Phrasal Rewrites with Large Language Models for ASR Post Editing

Hao Zhang, Felix Stahlberg, Shankar Kumar

TL;DR

The paper tackles the high computational cost of LLM-based rewriting by exploiting input-output overlap through compact edit representations in ASR post-editing. It introduces three representations—edit span, phrase pair, and target phrase—to encode rewrites efficiently and deterministically expand them back to full rewrites. Experiments on LibriSpeech using PaLM 2 variants show the target-phrase-only representation offers the best efficiency-accuracy trade-off, closing a substantial portion of the WER gap with modest decoding slowdown. This approach advances practical deployment of LLM-based post-editing for ASR by balancing accuracy and inference cost, and points to learnable, even more efficient compact representations in the future.

Abstract

Large Language Models (LLMs) excel at rewriting tasks such as text style transfer and grammatical error correction. While there is considerable overlap between the inputs and outputs in these tasks, the decoding cost still increases with output length, regardless of the amount of overlap. By leveraging the overlap between the input and the output, Kaneko and Okazaki (2023) proposed model-agnostic edit span representations to compress the rewrites to save computation. They reported an output length reduction rate of nearly 80% with minimal accuracy impact in four rewriting tasks. In this paper, we propose alternative edit phrase representations inspired by phrase-based statistical machine translation. We systematically compare our phrasal representations with their span representations. We apply the LLM rewriting model to the task of Automatic Speech Recognition (ASR) post editing and show that our target-phrase-only edit representation has the best efficiency-accuracy trade-off. On the LibriSpeech test set, our method closes 50-60% of the WER gap between the edit span model and the full rewrite model while losing only 10-20% of the length reduction rate of the edit span model.

Predicting Compact Phrasal Rewrites with Large Language Models for ASR Post Editing

TL;DR

The paper tackles the high computational cost of LLM-based rewriting by exploiting input-output overlap through compact edit representations in ASR post-editing. It introduces three representations—edit span, phrase pair, and target phrase—to encode rewrites efficiently and deterministically expand them back to full rewrites. Experiments on LibriSpeech using PaLM 2 variants show the target-phrase-only representation offers the best efficiency-accuracy trade-off, closing a substantial portion of the WER gap with modest decoding slowdown. This approach advances practical deployment of LLM-based post-editing for ASR by balancing accuracy and inference cost, and points to learnable, even more efficient compact representations in the future.

Abstract

Large Language Models (LLMs) excel at rewriting tasks such as text style transfer and grammatical error correction. While there is considerable overlap between the inputs and outputs in these tasks, the decoding cost still increases with output length, regardless of the amount of overlap. By leveraging the overlap between the input and the output, Kaneko and Okazaki (2023) proposed model-agnostic edit span representations to compress the rewrites to save computation. They reported an output length reduction rate of nearly 80% with minimal accuracy impact in four rewriting tasks. In this paper, we propose alternative edit phrase representations inspired by phrase-based statistical machine translation. We systematically compare our phrasal representations with their span representations. We apply the LLM rewriting model to the task of Automatic Speech Recognition (ASR) post editing and show that our target-phrase-only edit representation has the best efficiency-accuracy trade-off. On the LibriSpeech test set, our method closes 50-60% of the WER gap between the edit span model and the full rewrite model while losing only 10-20% of the length reduction rate of the edit span model.
Paper Structure (11 sections, 4 equations, 3 figures, 3 tables)

This paper contains 11 sections, 4 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Rewrite Representations
  3. Edit Span Representation
  4. Phrase Pair Representation
  5. Target Phrase Representation
  6. Experiments
  7. Efficiency and Accuracy Trade-offs
  8. Recovery Rate of Compact Representations
  9. Related Work
  10. Limitations
  11. Conclusions

Figures (3)

  • Figure 1: Visualization of the span, phrase pair, and target phrase representations.
  • Figure 2: Overview of the entire ASR system with the edit-representation-based LLM post editor.
  • Figure 3: WER versus output length on the dev set. Left: PaLM 2 Gecko model. Right: PaLM 2 Otter model. target only with dilation span size 3 (the third ${ \bigtriangleup}$ from the left) is the best strategy.