Using External knowledge to Enhanced PLM for Semantic Matching

TL;DR

This paper addresses semantic relevance modeling (SRM) by integrating external lexical knowledge into pre-trained language models to improve sentence-pair understanding. It introduces a knowledge-infused attention framework that constructs a prior knowledge matrix from relations such as synonyms, antonyms, hypernyms, and hyponyms (e.g., via WordNet), and calibrates self-attention using an adaptive fusion module that jointly attends to semantic and knowledge signals with gating and filtration mechanisms. Across ten public datasets, including GLUE tasks and robustness benchmarks, the approach yields consistent improvements over strong baselines like BERT and knowledge-enhanced variants, with notable gains on QQP and enhanced robustness to perturbations. The contributions offer improved SRM accuracy and interpretability through explicit knowledge fusion, advancing practical semantic matching in NLP applications.

Abstract

Modeling semantic relevance has always been a challenging and critical task in natural language processing. In recent years, with the emergence of massive amounts of annotated data, it has become feasible to train complex models, such as neural network-based reasoning models. These models have shown excellent performance in practical applications and have achieved the current state-ofthe-art performance. However, even with such large-scale annotated data, we still need to think: Can machines learn all the knowledge necessary to perform semantic relevance detection tasks based on this data alone? If not, how can neural network-based models incorporate external knowledge into themselves, and how can relevance detection models be constructed to make full use of external knowledge? In this paper, we use external knowledge to enhance the pre-trained semantic relevance discrimination model. Experimental results on 10 public datasets show that our method achieves consistent improvements in performance compared to the baseline model.