Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cross-Domain Robustness of Transformer-based Keyphrase Generation

Anna Glazkova, Dmitry Morozov

TL;DR

The experiments show that preliminary fine-tuning on out-of-domain corpora can be effective under conditions of a limited number of samples, and the role of transfer learning between different domains to improve the BART model performance on small text corpora is explored.

Abstract

Modern models for text generation show state-of-the-art results in many natural language processing tasks. In this work, we explore the effectiveness of abstractive text summarization models for keyphrase selection. A list of keyphrases is an important element of a text in databases and repositories of electronic documents. In our experiments, abstractive text summarization models fine-tuned for keyphrase generation show quite high results for a target text corpus. However, in most cases, the zero-shot performance on other corpora and domains is significantly lower. We investigate cross-domain limitations of abstractive text summarization models for keyphrase generation. We present an evaluation of the fine-tuned BART models for the keyphrase selection task across six benchmark corpora for keyphrase extraction including scientific texts from two domains and news texts. We explore the role of transfer learning between different domains to improve the BART model performance on small text corpora. Our experiments show that preliminary fine-tuning on out-of-domain corpora can be effective under conditions of a limited number of samples.

Cross-Domain Robustness of Transformer-based Keyphrase Generation

TL;DR

The experiments show that preliminary fine-tuning on out-of-domain corpora can be effective under conditions of a limited number of samples, and the role of transfer learning between different domains to improve the BART model performance on small text corpora is explored.

Abstract

Modern models for text generation show state-of-the-art results in many natural language processing tasks. In this work, we explore the effectiveness of abstractive text summarization models for keyphrase selection. A list of keyphrases is an important element of a text in databases and repositories of electronic documents. In our experiments, abstractive text summarization models fine-tuned for keyphrase generation show quite high results for a target text corpus. However, in most cases, the zero-shot performance on other corpora and domains is significantly lower. We investigate cross-domain limitations of abstractive text summarization models for keyphrase generation. We present an evaluation of the fine-tuned BART models for the keyphrase selection task across six benchmark corpora for keyphrase extraction including scientific texts from two domains and news texts. We explore the role of transfer learning between different domains to improve the BART model performance on small text corpora. Our experiments show that preliminary fine-tuning on out-of-domain corpora can be effective under conditions of a limited number of samples.
Paper Structure (10 sections, 3 figures, 6 tables)

This paper contains 10 sections, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Abstractive Text Summarization using Pre-trained Transformers
  4. Keyword Selection
  5. Data
  6. Models
  7. Experimental Setup
  8. Results and Discussion
  9. Conclusion
  10. Appendix

Figures (3)

  • Figure 1: An example of a source text with the corresponding list of from the Inspec corpus hulth2003improved. The keyphrases that appear in the text are underlined.
  • Figure 2: Adding training examples from other corpora and domains.
  • Figure 3: Performance of BART with and without preliminary fine-tuning on out-of-domain corpora using a varying size of training data.