G-SAP: Graph-based Structure-Aware Prompt Learning over Heterogeneous Knowledge for Commonsense Reasoning

TL;DR

A novel G-SAP is proposed aiming to maintain a balance between heterogeneous knowledge and enhance the cross-modal interaction within the LM+GNNs model, aiming to maintain a balance between heterogeneous knowledge and enhance the cross-modal interaction within the LM+GNNs model.

Abstract

Commonsense question answering has demonstrated considerable potential across various applications like assistants and social robots. Although fully fine-tuned pre-trained Language Models(LM) have achieved remarkable performance in commonsense reasoning, their tendency to excessively prioritize textual information hampers the precise transfer of structural knowledge and undermines interpretability. Some studies have explored combining LMs with Knowledge Graphs(KGs) by coarsely fusing the two modalities to perform Graph Neural Network(GNN)-based reasoning that lacks a profound interaction between heterogeneous modalities. In this paper, we propose a novel Graph-based Structure-Aware Prompt Learning Model for commonsense reasoning, named G-SAP, aiming to maintain a balance between heterogeneous knowledge and enhance the cross-modal interaction within the LM+GNNs model. In particular, an evidence graph is constructed by integrating multiple knowledge sources, i.e. ConceptNet, Wikipedia, and Cambridge Dictionary to boost the performance. Afterward, a structure-aware frozen PLM is employed to fully incorporate the structured and textual information from the evidence graph, where the generation of prompts is driven by graph entities and relations. Finally, a heterogeneous message-passing reasoning module is used to facilitate deep interaction of knowledge between the LM and graph-based networks. Empirical validation, conducted through extensive experiments on three benchmark datasets, demonstrates the notable performance of the proposed model. The results reveal a significant advancement over the existing models, especially, with 6.12% improvement over the SoTA LM+GNNs model on the OpenbookQA dataset.

Figures (3)

• Figure 1: Comparison of existing methods and our G-SAP for CSQA. (a) PLM-based methods feed QA content, or together with the retrieved knowledge into LM for full fine-tuning. (b) LM+GNNs methods leverage shallow cross-modal interaction operations to fuse the encoded representations from LM and GNN. (c) Our G-SAP uses structure-aware prompts to fully interact KG structure information with textual information in PLM (SAPL). The outputs are then fed into HMPS to update the KG graph and textual representation for heterogeneous message-passing reasoning.
• Figure 2: Overall framework of G-SAP. We first extract evidence from sources to obtain an emerged evidence graph, then introduce paraphrase and relevance scores to refine it. Secondly, structure-aware prompts are constructed based on KG nodes and relations representations, enabling the inclusion of both structural and textual information in the frozen PLMs. Finally, a heterogeneous messaging-passing strategy is employed, utilizing graph-based attention networks that fuse the PLM results to iteratively update the evidence graph representation and generate the final answer prediction.
• Figure 3: The impact of prompt length