A Simple Contrastive Learning Objective for Alleviating Neural Text Degeneration

TL;DR

The paper tackles neural text degeneration in autoregressive language models trained with cross-entropy by introducing Contrastive Token Learning (CT), a token-level objective that combines CE with a focused contrastive loss comparing positives to a small set of negative tokens drawn from preceding context. CT explicitly promotes label tokens while suppressing frequent negative tokens, leaving irrelevant tokens untouched, which reduces repetition without drastically harming perplexity. Experiments on language modeling and open-domain dialogue show CT yields substantially less repetitive text and higher generation quality, outperforming prior unlikelihood and contrastive baselines and achieving state-of-the-art degeneration mitigation. The approach offers a simple, general training objective with potential to improve a wide range of generation tasks while maintaining compatibility with existing decoding strategies.

Abstract

The cross-entropy objective has proved to be an all-purpose training objective for autoregressive language models (LMs). However, without considering the penalization of problematic tokens, LMs trained using cross-entropy exhibit text degeneration. To address this, unlikelihood training has been proposed to reduce the probability of unlikely tokens predicted by LMs. But unlikelihood does not consider the relationship between the label tokens and unlikely token candidates, thus showing marginal improvements in degeneration. We propose a new contrastive token learning objective that inherits the advantages of cross-entropy and unlikelihood training and avoids their limitations. The key idea is to teach a LM to generate high probabilities for label tokens and low probabilities of negative candidates. Comprehensive experiments on language modeling and open-domain dialogue generation tasks show that the proposed contrastive token objective yields much less repetitive texts, with a higher generation quality than baseline approaches, achieving the new state-of-the-art performance on text degeneration.

Figures (11)

• Figure 1: Illustrating the differences between our proposed contrastive token learning, unlikelihood training, and the cross-entropy objective for LMs. For contrastive token learning, we use the label token as the positive token and the preceding $M$ tokens as the negative tokens at each decoding step.
• Figure 2: Histograms for rep-1 (left) and rep-4 (right) rates of each method, on the Wikitext-103 test set.
• Figure 3: Heat maps for the generation probability of CT, CE and, UL-TS, at inference time. Row and column labels represent model-generated tokens at each time step, and the saturation of each cell represents the corresponding probability of each token. Please refer to §\ref{['sec:prob']} for a more detailed description. Heat maps for NCE, UL-T and SimCTG look similar to that of CE, and can be found in Appendix F, Figure 4.
• Figure 4: Heat maps for the generation probability of NCE, UL-T and SimCTG on the Wikitext-103 test set.
• Figure 5: Influence of the sequence length for CT loss on the language modeling task.