Modeling Comparative Logical Relation with Contrastive Learning for Text Generation

TL;DR

The paper addresses the limited ability of Data-to-Text Generation (D2T) systems to verbalize comparative logical relations (CLRs) between entities. It introduces the CLRG task and the Chinese CLR Dataset (CLRD), and presents the CoLo framework, a two-stage contrastive learning approach with Contrastive Encoding and Contrastive Decoding, using positive and negative CLR perturbations to guide learning. The model demonstrates strong performance on automatic and human evaluations, outperforming several baselines with only 0.58B parameters and approaching GPT-3.5 on entailment and coverage, while providing high-quality CLR descriptions for e-commerce style texts. This work advances the ability to generate fluent, logically coherent text that accurately expresses comparative relations, with potential impact on multilingual, logic-aware NLG for real-world applications.

Abstract

Data-to-Text Generation (D2T), a classic natural language generation problem, aims at producing fluent descriptions for structured input data, such as a table. Existing D2T works mainly focus on describing the superficial associative relations among entities, while ignoring the deep comparative logical relations, such as A is better than B in a certain aspect with a corresponding opinion, which is quite common in our daily life. In this paper, we introduce a new D2T task named comparative logical relation generation (CLRG). Additionally, we propose a Comparative Logic (CoLo) based text generation method, which generates texts following specific comparative logical relations with contrastive learning. Specifically, we first construct various positive and negative samples by fine-grained perturbations in entities, aspects and opinions. Then, we perform contrastive learning in the encoder layer to have a better understanding of the comparative logical relations, and integrate it in the decoder layer to guide the model to correctly generate the relations. Noting the data scarcity problem, we construct a Chinese Comparative Logical Relation Dataset (CLRD), which is a high-quality human-annotated dataset and challenging for text generation with descriptions of multiple entities and annotations on their comparative logical relations. Extensive experiments show that our method achieves impressive performance in both automatic and human evaluations.

Figures (5)

• Figure 1: Examples from two popular Data-to-Text datasets and the proposed Chinese Comparative Logical Relation Dataset (CLRD): (a) Descriptions based on a single entity with attributes; (b) Descriptions with an associative relation (e.g., "directed by"); (c) Descriptions with a comparative logical relation, including aspects, opinions, and order. Entities and relations are color-coded.
• Figure 2: Overview of our proposed method. On the left, our strategy for constructing contrastive samples is shown. On the right, we introduce two contrastive losses beyond the standard language modeling loss ($\mathcal{L}_{\mathrm{LM}}$): the contrastive encoding loss ($\mathcal{L}_{\mathrm{CE}}$) and the contrastive decoding loss ($\mathcal{L}_{\mathrm{CD}}$).
• Figure 3: Examples of an original tuple and its contrastive tuples. The positive tuple is derived by replacing words in the original tuple with their synonyms. The negative tuples are constructed by entity swapping, aspect substitution and opinion substitution. Details can be found in Section \ref{['sec:cec']}.
• Figure 4: Results of ablation studies. - $\mathcal{L}_{\mathrm{CE}}$ excludes $\mathcal{L}_{\mathrm{CE}}$ from Colo. - $\mathcal{L}_{\mathrm{CD}}$ removes $\mathcal{L}_{\mathrm{CD}}$ from Colo. - both removes $\mathcal{L}_{\mathrm{CE}}$ and $\mathcal{L}_{\mathrm{CD}}$ from Colo.
• Figure 5: Descriptions generated by our model and baselines.