Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UniGen: Universal Domain Generalization for Sentiment Classification via Zero-shot Dataset Generation

Juhwan Choi, Yeonghwa Kim, Seunguk Yu, JungMin Yun, YoungBin Kim

TL;DR

A novel approach to universal domain generalization that generates a dataset regardless of the target domain regardless of the target domain is proposed that accomplishes generalizability across various domains while using a parameter set that is orders of magnitude smaller than PLMs.

Abstract

Although pre-trained language models have exhibited great flexibility and versatility with prompt-based few-shot learning, they suffer from the extensive parameter size and limited applicability for inference. Recent studies have suggested that PLMs be used as dataset generators and a tiny task-specific model be trained to achieve efficient inference. However, their applicability to various domains is limited because they tend to generate domain-specific datasets. In this work, we propose a novel approach to universal domain generalization that generates a dataset regardless of the target domain. This allows for generalization of the tiny task model to any domain that shares the label space, thus enhancing the real-world applicability of the dataset generation paradigm. Our experiments indicate that the proposed method accomplishes generalizability across various domains while using a parameter set that is orders of magnitude smaller than PLMs.

UniGen: Universal Domain Generalization for Sentiment Classification via Zero-shot Dataset Generation

TL;DR

A novel approach to universal domain generalization that generates a dataset regardless of the target domain regardless of the target domain is proposed that accomplishes generalizability across various domains while using a parameter set that is orders of magnitude smaller than PLMs.

Abstract

Although pre-trained language models have exhibited great flexibility and versatility with prompt-based few-shot learning, they suffer from the extensive parameter size and limited applicability for inference. Recent studies have suggested that PLMs be used as dataset generators and a tiny task-specific model be trained to achieve efficient inference. However, their applicability to various domains is limited because they tend to generate domain-specific datasets. In this work, we propose a novel approach to universal domain generalization that generates a dataset regardless of the target domain. This allows for generalization of the tiny task model to any domain that shares the label space, thus enhancing the real-world applicability of the dataset generation paradigm. Our experiments indicate that the proposed method accomplishes generalizability across various domains while using a parameter set that is orders of magnitude smaller than PLMs.
Paper Structure (28 sections, 5 equations, 2 figures, 11 tables)

This paper contains 28 sections, 5 equations, 2 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dataset Generation for Efficient Zero-shot Learning
  4. Methods for Learning from Noisy Data
  5. Domain Generalization for Text Classification
  6. Method
  7. Preliminaries
  8. Dataset Generation
  9. Supervised Contrastive Learning
  10. UniGen
  11. Handling Noisy Data through Relabeling
  12. Denoising Memory Bank
  13. Experiment
  14. Experimental Setup
  15. Comparison with Task-specific TAMs
  16. ...and 13 more sections

Figures (2)

  • Figure 1: Overall framework for generating a dataset and training a TAM using UniGen.
  • Figure 2: T-SNE visualization of the encoded representation of the RoBERTa model trained using UniGen. The model was trained only on the data generated using UniGen, which is shown in gray color. We used the test set of the multi-domain review dataset.