Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CO3: Low-resource Contrastive Co-training for Generative Conversational Query Rewrite

Yifei Yuan, Chen Shi, Runze Wang, Liyi Chen, Renjun Hu, Zengming Zhang, Feijun Jiang, Wai Lam

TL;DR

This work co-train two dual models such that each of them provides extra guidance through pseudo-labeling for enhancing the other in an iterative manner to utilize massive unlabeled data to make further improvements via a contrastive co-training paradigm.

Abstract

Generative query rewrite generates reconstructed query rewrites using the conversation history while rely heavily on gold rewrite pairs that are expensive to obtain. Recently, few-shot learning is gaining increasing popularity for this task, whereas these methods are sensitive to the inherent noise due to limited data size. Besides, both attempts face performance degradation when there exists language style shift between training and testing cases. To this end, we study low-resource generative conversational query rewrite that is robust to both noise and language style shift. The core idea is to utilize massive unlabeled data to make further improvements via a contrastive co-training paradigm. Specifically, we co-train two dual models (namely Rewriter and Simplifier) such that each of them provides extra guidance through pseudo-labeling for enhancing the other in an iterative manner. We also leverage contrastive learning with data augmentation, which enables our model pay more attention on the truly valuable information than the noise. Extensive experiments demonstrate the superiority of our model under both few-shot and zero-shot scenarios. We also verify the better generalization ability of our model when encountering language style shift.

CO3: Low-resource Contrastive Co-training for Generative Conversational Query Rewrite

TL;DR

This work co-train two dual models such that each of them provides extra guidance through pseudo-labeling for enhancing the other in an iterative manner to utilize massive unlabeled data to make further improvements via a contrastive co-training paradigm.

Abstract

Generative query rewrite generates reconstructed query rewrites using the conversation history while rely heavily on gold rewrite pairs that are expensive to obtain. Recently, few-shot learning is gaining increasing popularity for this task, whereas these methods are sensitive to the inherent noise due to limited data size. Besides, both attempts face performance degradation when there exists language style shift between training and testing cases. To this end, we study low-resource generative conversational query rewrite that is robust to both noise and language style shift. The core idea is to utilize massive unlabeled data to make further improvements via a contrastive co-training paradigm. Specifically, we co-train two dual models (namely Rewriter and Simplifier) such that each of them provides extra guidance through pseudo-labeling for enhancing the other in an iterative manner. We also leverage contrastive learning with data augmentation, which enables our model pay more attention on the truly valuable information than the noise. Extensive experiments demonstrate the superiority of our model under both few-shot and zero-shot scenarios. We also verify the better generalization ability of our model when encountering language style shift.
Paper Structure (34 sections, 9 equations, 4 figures, 8 tables, 1 algorithm)

This paper contains 34 sections, 9 equations, 4 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Conversational Query Rewrite
  4. Co-training Paradigm
  5. Our Framework
  6. Problem Formulation
  7. Framework Overview
  8. Co-training Paradigm
  9. Contrastive Data Augmentation
  10. Internal and External Contrastive Loss
  11. Contrastive Loss Function
  12. Training
  13. Experiments
  14. Datasets
  15. Labeled Dataset.
  16. ...and 19 more sections

Figures (4)

  • Figure 1: The overall framework of our proposed paradigm.
  • Figure 2: Performance of CO3 in each iteration.
  • Figure 3: Real case and error case analysis. The first three are examples under the few-shot setting and the last four are under the zero-shot setting. The blue part denotes the resolved coreference or completed ellipsis in the gold rewrite. The red part denotes the errors in the model output.
  • Figure 4: The upper part is the weakly-labeled data weight analysis and the lower part is the contrastive weight analysis.