Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Heterogeneous Graph Reasoning for Fact Checking over Texts and Tables

Haisong Gong, Weizhi Xu, Shu wu, Qiang Liu, Liang Wang

TL;DR

HeterFC tackles veracity reasoning over blended unstructured text and structured tables by constructing a word-level heterogeneous evidence graph with intra-sentence, intra-table, and inter-evidence edges. Information is propagated via Relational Graph Convolutional Networks, followed by an attention-based interaction with the claim and a PLM-assisted prediction, trained with a multitask loss to mitigate noisy evidence retrieval. The approach yields state-of-the-art results on FEVEROUS, outperforming both transformer-based and homogeneous-graph baselines, and ablations validate the importance of word-level granularity, heterogeneous edges, and the attention-assisted readout. This work offers a robust, scalable framework for multi-evidence reasoning in fact checking, with potential impact on real-world verification systems across text and tabular data streams.

Abstract

Fact checking aims to predict claim veracity by reasoning over multiple evidence pieces. It usually involves evidence retrieval and veracity reasoning. In this paper, we focus on the latter, reasoning over unstructured text and structured table information. Previous works have primarily relied on fine-tuning pretrained language models or training homogeneous-graph-based models. Despite their effectiveness, we argue that they fail to explore the rich semantic information underlying the evidence with different structures. To address this, we propose a novel word-level Heterogeneous-graph-based model for Fact Checking over unstructured and structured information, namely HeterFC. Our approach leverages a heterogeneous evidence graph, with words as nodes and thoughtfully designed edges representing different evidence properties. We perform information propagation via a relational graph neural network, facilitating interactions between claims and evidence. An attention-based method is utilized to integrate information, combined with a language model for generating predictions. We introduce a multitask loss function to account for potential inaccuracies in evidence retrieval. Comprehensive experiments on the large fact checking dataset FEVEROUS demonstrate the effectiveness of HeterFC. Code will be released at: https://github.com/Deno-V/HeterFC.

Heterogeneous Graph Reasoning for Fact Checking over Texts and Tables

TL;DR

HeterFC tackles veracity reasoning over blended unstructured text and structured tables by constructing a word-level heterogeneous evidence graph with intra-sentence, intra-table, and inter-evidence edges. Information is propagated via Relational Graph Convolutional Networks, followed by an attention-based interaction with the claim and a PLM-assisted prediction, trained with a multitask loss to mitigate noisy evidence retrieval. The approach yields state-of-the-art results on FEVEROUS, outperforming both transformer-based and homogeneous-graph baselines, and ablations validate the importance of word-level granularity, heterogeneous edges, and the attention-assisted readout. This work offers a robust, scalable framework for multi-evidence reasoning in fact checking, with potential impact on real-world verification systems across text and tabular data streams.

Abstract

Fact checking aims to predict claim veracity by reasoning over multiple evidence pieces. It usually involves evidence retrieval and veracity reasoning. In this paper, we focus on the latter, reasoning over unstructured text and structured table information. Previous works have primarily relied on fine-tuning pretrained language models or training homogeneous-graph-based models. Despite their effectiveness, we argue that they fail to explore the rich semantic information underlying the evidence with different structures. To address this, we propose a novel word-level Heterogeneous-graph-based model for Fact Checking over unstructured and structured information, namely HeterFC. Our approach leverages a heterogeneous evidence graph, with words as nodes and thoughtfully designed edges representing different evidence properties. We perform information propagation via a relational graph neural network, facilitating interactions between claims and evidence. An attention-based method is utilized to integrate information, combined with a language model for generating predictions. We introduce a multitask loss function to account for potential inaccuracies in evidence retrieval. Comprehensive experiments on the large fact checking dataset FEVEROUS demonstrate the effectiveness of HeterFC. Code will be released at: https://github.com/Deno-V/HeterFC.
Paper Structure (30 sections, 9 equations, 4 figures, 3 tables)

This paper contains 30 sections, 9 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Fact Checking Over Unstructured Information
  4. Fact Checking Over Mixed-type Information
  5. Heterogeneous Graph Neural Networks
  6. Method
  7. Task Formulation
  8. Word-level Evidence Graph Construction
  9. Node Construction
  10. Edge Construction
  11. Heterogeneous Information Propagation
  12. Evidence-level Representation Readout
  13. Attention-based Claim-evidence Interaction
  14. Fused Veracity Prediction
  15. Model Training
  16. ...and 15 more sections

Figures (4)

  • Figure 1: Architecture of HeterFC. Inputs are claim $c$ and an evidence set $\mathcal{E}$. There are five main parts: 1) Word-level evidence graph construction. Initial node embeddings are obtained by using PLM and subword mean pooling. Three types of edges are designed for the heterogeneous connection. 2) Heterogeneous information propagation. R-GCN is used to perform neighborhood aggregation on the word-level evidence graph. 3) Evidence-level representation readout. Evidence representations are obtained by pooling over subgraphs corresponding to each piece of evidence. 4) Attention-based claim-evidence interaction. Graph representation $\mathbf{o}_g$ is generated by claim-guided evidence combination. A supervised loss item, $Loss_{e}$, is computed based on the attention assignment. 5) Fused veracity prediction. The claim and evidence are concatenated and fed into PLM to obtain $\mathbf{o}_t$, which, when combined with the graph representation $\mathbf{o}_g$, forms the final representation. A fully-connected network takes the representation as input and generates the prediction $\mathbf{\hat{p}}$. HeterFC is trained using classification loss $Loss_c$ and assisted $Loss_e$.
  • Figure 2: The performance comparison among models with different graph granularity. $\text{HeterFC\xspace}_{\text{sent}}$ represents the variant with sentence-level graph, while $\text{HeterFC\xspace}_{\text{token}}$ represents the variant with token-level graph.
  • Figure 3: The performance comparison between the proposed model HeterFC and its three variants: $\text{HeterFC\xspace}_{\text{w/o gl}}$ ignores global and local connection designs. $\text{HeterFC\xspace}_{\text{w/o Het}}$ ignores heterogeneous relations. $\text{HeterFC\xspace}_{\text{w/o both}}$ removes all special designs for a fully-connected homogeneous graph.
  • Figure 4: The influence of hyper-parameters on HeterFC's performance on the development set.