Enhancing Automatic Keyphrase Labelling with Text-to-Text Transfer Transformer (T5) Architecture: A Framework for Keyphrase Generation and Filtering

TL;DR

The paper tackles automatic keyphrase generation (AKG) and filtering by leveraging the Text-to-Text Transfer Transformer (T5) framework. It introduces two main contributions: docT5keywords, a T5-based model that generates keyphrases from a document’s title and abstract, and keyFilT5r, a monoT5-style filter that assesses keyphrase relevance to curb hallucinations. Through extensive experiments on multiple datasets (Inspec, KP20k, KP-BioMed, MAG, KPTimes) and evaluation regimes (exact and partial matches for present/absent keyphrases, plus a binary filtering task), the study shows that docT5keywords substantially outperforms baselines, with additional gains from majority voting and selective filtering, though filtering is less effective for absent keyphrases. The results underscore the potential of combining generative AKG with a relevance-filtering step to improve trustworthiness and precision in real-world applications. Overall, the work demonstrates strong AKG performance with practical techniques to mitigate generation errors and hallucinations, advancing deployable keyphrase labelling systems.

Abstract

Automatic keyphrase labelling stands for the ability of models to retrieve words or short phrases that adequately describe documents' content. Previous work has put much effort into exploring extractive techniques to address this task; however, these methods cannot produce keyphrases not found in the text. Given this limitation, keyphrase generation approaches have arisen lately. This paper presents a keyphrase generation model based on the Text-to-Text Transfer Transformer (T5) architecture. Having a document's title and abstract as input, we learn a T5 model to generate keyphrases which adequately define its content. We name this model docT5keywords. We not only perform the classic inference approach, where the output sequence is directly selected as the predicted values, but we also report results from a majority voting approach. In this approach, multiple sequences are generated, and the keyphrases are ranked based on their frequency of occurrence across these sequences. Along with this model, we present a novel keyphrase filtering technique based on the T5 architecture. We train a T5 model to learn whether a given keyphrase is relevant to a document. We devise two evaluation methodologies to prove our model's capability to filter inadequate keyphrases. First, we perform a binary evaluation where our model has to predict if a keyphrase is relevant for a given document. Second, we filter the predicted keyphrases by several AKG models and check if the evaluation scores are improved. Experimental results demonstrate that our keyphrase generation model significantly outperforms all the baselines, with gains exceeding 100\% in some cases. The proposed filtering technique also achieves near-perfect accuracy in eliminating false positives across all datasets.

Figures (4)

• Figure 1: Fine-tuning T5 for keyphrase generation. To anticipate the keyphrases, both the task description and the document tokens are fed into the Transformer encoder.
• Figure 2: Fine-tuning T5 for keyphrase filtering. Both the keyphrase and the document it refers to are fed into the Transformer encoder along with the task descriptors to classify it as relevant or irrelevant.
• Figure 3: Example of a keyphrase co-occurrence graph with its connected components.
• Figure 4: keyFilT5r binary evaluation accuracy confusion matrices.