Multi-Evidence based Fact Verification via A Confidential Graph Neural Network

TL;DR

CO-GAT introduces a Confidential Graph Attention Network for fact verification that pre-emptively masks noisy evidence before graph reasoning using a node confidence score (CO-SCO) computed from claim–evidence relevance. The masked node representations are then propagated via a fully connected graph with multi-head edge attention and node attention to predict claim labels, trained with a dual objective L = L_fact + L_evi. Empirical results on FEVER and SCIFACT show strong FEVER scores (notably 73.59% on the blind test) and improvements over baselines, with ablations confirming the utility of node masking and multi-task learning. The approach improves robustness to noise and demonstrates generalization to science-domain claims, offering practical gains for automated fact verification systems.

Abstract

Fact verification tasks aim to identify the integrity of textual contents according to the truthful corpus. Existing fact verification models usually build a fully connected reasoning graph, which regards claim-evidence pairs as nodes and connects them with edges. They employ the graph to propagate the semantics of the nodes. Nevertheless, the noisy nodes usually propagate their semantics via the edges of the reasoning graph, which misleads the semantic representations of other nodes and amplifies the noise signals. To mitigate the propagation of noisy semantic information, we introduce a Confidential Graph Attention Network (CO-GAT), which proposes a node masking mechanism for modeling the nodes. Specifically, CO-GAT calculates the node confidence score by estimating the relevance between the claim and evidence pieces. Then, the node masking mechanism uses the node confidence scores to control the noise information flow from the vanilla node to the other graph nodes. CO-GAT achieves a 73.59% FEVER score on the FEVER dataset and shows the generalization ability by broadening the effectiveness to the science-specific domain.

Figures (5)

• Figure 1: An Example of Fact Verification System. $E_1$ represents the golden evidence of the given claim. The remained evidence nodes indicates that the claim and evidence piece are irrelevant. $E_b$ represents the blank node. The given claim and each piece of evidence are concatenated as graph nodes. Nodes are connected via edges, forming a fully connected graph for claim verification reasoning.
• Figure 2: Illustration of CO-GAT. It is mainly divided into three steps. Firstly, the pre-trained language models are used to encode the graph nodes, which are claim-evidence pairs. Secondly, the node masking mechanism is utilized to erase noise information based on the confidence score (CO-SCO) before graph reasoning. Finally, we use a fully connected graph to propagate the denoised evidence information among nodes and aggregate node information for predicting the claim label.
• Figure 3: Tendency and Distribution of NEI Label Prediction. Fig. \ref{['fig：nei_all']} represents the prediction probability of NEI changing with cross-entropy. The increase of cross entropy indicates the higher uncertainty of the model DBLP:conf/emnlp/WangLWLS19DBLP:conf/icml/OttAGR18. Fig. \ref{['fig:nei_false']} represents the NEI distribution in the cases that make a mistake prediction.
• Figure 4: The Effectiveness of Confidence Score in Controlling Information Flows from Encoded Node Representations to Graph Reasoning. Fig. \ref{['fig_second_case']} describes the change in the probability of predicting the label as NEI. Fig. \ref{['fig_first_case']} describes the tendency of label accuracy. Fig. \ref{['fig_third_case']} describes the tendency of edge attention entropy and node attention entropy. We scale the CO-SCO by multiplying the original confidence score using a coefficient $\alpha \in [0.0, 1.0]$.
• Figure 5: The Behaviors of Edge Attention and Node Attention After Adding Node Masking Mechanism. Fig. \ref{['fig: 4a']} shows the attention entropy of the edge and the node of the CO-GAT model and the MHA-GAT model, respectively. Fig. \ref{['fig: 4b']} shows the edge attention weight distribution of the CO-GAT model and the MHA-GAT model. Fig. \ref{['fig: 4c']} shows the node attention weight distribution of the CO-GAT model and the MHA-GAT model.