Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Classical Structured Prediction Losses for Sequence to Sequence Learning

Sergey Edunov, Myle Ott, Michael Auli, David Grangier, Marc'Aurelio Ranzato

TL;DR

The paper tackles the challenge of training neural sequence-to-sequence models at the sequence level by revisiting classical structured prediction losses. It systematically compares token-level losses (e.g., TokNLL, TokLS) with sequence-level losses (e.g., SeqNLL, Risk, MaxMargin, MultiMargin, SoftmaxMargin) and explores combinations, candidate-generation strategies, and initialization effects. The findings show that sequence-level losses outperform token-level training, that combining token- and sequence-level objectives yields the best results, and that risk-based sequence losses can match or exceed beam-search optimization on several tasks, including IWSLT'14, Gigaword, and WMT'14 EN-FR. Overall, the study demonstrates that classical structured-prediction losses remain highly effective for neural seq2seq models and offers practical guidance on loss design and candidate generation for improved generation quality.

Abstract

There has been much recent work on training neural attention models at the sequence-level using either reinforcement learning-style methods or by optimizing the beam. In this paper, we survey a range of classical objective functions that have been widely used to train linear models for structured prediction and apply them to neural sequence to sequence models. Our experiments show that these losses can perform surprisingly well by slightly outperforming beam search optimization in a like for like setup. We also report new state of the art results on both IWSLT'14 German-English translation as well as Gigaword abstractive summarization. On the larger WMT'14 English-French translation task, sequence-level training achieves 41.5 BLEU which is on par with the state of the art.

Classical Structured Prediction Losses for Sequence to Sequence Learning

TL;DR

The paper tackles the challenge of training neural sequence-to-sequence models at the sequence level by revisiting classical structured prediction losses. It systematically compares token-level losses (e.g., TokNLL, TokLS) with sequence-level losses (e.g., SeqNLL, Risk, MaxMargin, MultiMargin, SoftmaxMargin) and explores combinations, candidate-generation strategies, and initialization effects. The findings show that sequence-level losses outperform token-level training, that combining token- and sequence-level objectives yields the best results, and that risk-based sequence losses can match or exceed beam-search optimization on several tasks, including IWSLT'14, Gigaword, and WMT'14 EN-FR. Overall, the study demonstrates that classical structured-prediction losses remain highly effective for neural seq2seq models and offers practical guidance on loss design and candidate generation for improved generation quality.

Abstract

There has been much recent work on training neural attention models at the sequence-level using either reinforcement learning-style methods or by optimizing the beam. In this paper, we survey a range of classical objective functions that have been widely used to train linear models for structured prediction and apply them to neural sequence to sequence models. Our experiments show that these losses can perform surprisingly well by slightly outperforming beam search optimization in a like for like setup. We also report new state of the art results on both IWSLT'14 German-English translation as well as Gigaword abstractive summarization. On the larger WMT'14 English-French translation task, sequence-level training achieves 41.5 BLEU which is on par with the state of the art.

Paper Structure

This paper contains 20 sections, 6 equations, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Sequence to Sequence Learning
  3. Objective Functions
  4. Token-Level Objectives
  5. Sequence-Level Objectives
  6. Combined Objectives
  7. Candidate Generation Strategies
  8. Experimental Setup
  9. Translation
  10. Abstractive Summarization
  11. Results
  12. Comparison of Sequence Level Losses
  13. Combination with Token-Level Loss
  14. Effect of initialization
  15. Online vs. Offline Candidate Generation
  16. ...and 5 more sections

Figures (2)

  • Figure 1: Token and sequence negative log-likelihood (Equations \ref{['eq:toknll']} and \ref{['eq:seqnll']}), token-level label smoothing (Equation \ref{['eq:tokls']}), expected risk (Equation \ref{['eq:risk']}), max-margin (Equation \ref{['eq:maxmargin']}), multi-margin (Equation \ref{['eq:multimargin']}), softmax-margin (Equation \ref{['eq:smm']}). We denote the source as $\mathbf{x}$, the reference target as $\mathbf{t}$, the set of candidate outputs as $\mathcal{U}$ and the best candidate (pseudo reference) as $\mathbf{u}^*$. For max-margin we denote the candidate with the highest model score as $\hat{\mathbf{u}}$.
  • Figure 2: Candidate set generation with beam search and sampling for various candidate set sizes during sequence-level training in terms of validation accuracy. Token-level label smoothing (TokLS) is the baseline.