Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AMREx: AMR for Explainable Fact Verification

Chathuri Jayaweera, Sangpil Youm, Bonnie Dorr

TL;DR

AMREx introduces an Abstract Meaning Representation–based approach to veracity prediction that outputs partially explainable justifications via AMR node mappings. The system combines AMR-based Smatch similarity with SBERT cosine similarity to assess entailment between claims and evidence, aggregating results across evidences to determine veracity on FEVER and AVeriTeC. While not always beating strong baselines, AMREx provides transparent, traceable explanations and can be used to prompt LLMs to generate faithful natural-language explanations, enhancing diagnostic insight and reducing hallucinations. The work demonstrates the value of explainability in fact verification and highlights dataset-specific considerations for thresholds and evidence types, pointing to future improvements in AMR parsing and interpretability.

Abstract

With the advent of social media networks and the vast amount of information circulating through them, automatic fact verification is an essential component to prevent the spread of misinformation. It is even more useful to have fact verification systems that provide explanations along with their classifications to ensure accurate predictions. To address both of these requirements, we implement AMREx, an Abstract Meaning Representation (AMR)-based veracity prediction and explanation system for fact verification using a combination of Smatch, an AMR evaluation metric to measure meaning containment and textual similarity, and demonstrate its effectiveness in producing partially explainable justifications using two community standard fact verification datasets, FEVER and AVeriTeC. AMREx surpasses the AVeriTec baseline accuracy showing the effectiveness of our approach for real-world claim verification. It follows an interpretable pipeline and returns an explainable AMR node mapping to clarify the system's veracity predictions when applicable. We further demonstrate that AMREx output can be used to prompt LLMs to generate natural-language explanations using the AMR mappings as a guide to lessen the probability of hallucinations.

AMREx: AMR for Explainable Fact Verification

TL;DR

AMREx introduces an Abstract Meaning Representation–based approach to veracity prediction that outputs partially explainable justifications via AMR node mappings. The system combines AMR-based Smatch similarity with SBERT cosine similarity to assess entailment between claims and evidence, aggregating results across evidences to determine veracity on FEVER and AVeriTeC. While not always beating strong baselines, AMREx provides transparent, traceable explanations and can be used to prompt LLMs to generate faithful natural-language explanations, enhancing diagnostic insight and reducing hallucinations. The work demonstrates the value of explainability in fact verification and highlights dataset-specific considerations for thresholds and evidence types, pointing to future improvements in AMR parsing and interpretability.

Abstract

With the advent of social media networks and the vast amount of information circulating through them, automatic fact verification is an essential component to prevent the spread of misinformation. It is even more useful to have fact verification systems that provide explanations along with their classifications to ensure accurate predictions. To address both of these requirements, we implement AMREx, an Abstract Meaning Representation (AMR)-based veracity prediction and explanation system for fact verification using a combination of Smatch, an AMR evaluation metric to measure meaning containment and textual similarity, and demonstrate its effectiveness in producing partially explainable justifications using two community standard fact verification datasets, FEVER and AVeriTeC. AMREx surpasses the AVeriTec baseline accuracy showing the effectiveness of our approach for real-world claim verification. It follows an interpretable pipeline and returns an explainable AMR node mapping to clarify the system's veracity predictions when applicable. We further demonstrate that AMREx output can be used to prompt LLMs to generate natural-language explanations using the AMR mappings as a guide to lessen the probability of hallucinations.

Paper Structure

This paper contains 16 sections, 5 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. NLI for Fact Verification
  4. Explainable Representations on Fact Verification
  5. Abstract Meaning Representation (AMR)
  6. Experiment
  7. Datasets
  8. FEVER dataset
  9. AVeriTeC dataset
  10. AMREx Model
  11. NLI model
  12. Fact verification for FEVER
  13. Fact verification for AVeriTeC
  14. Results and Analyses
  15. Explainability of the Model
  16. ...and 1 more sections

Figures (7)

  • Figure 1: Explainable fact verification pipeline. Lower Left: AMR graph for the Claim, Lower Middle: AMR graph for the Evidence, Lower Right: The AMR graph mapping to explain the model's prediction as "Supports"
  • Figure 2: AMREx model: The model aggregates all the entailment predictions from the NLI model for a claim and returns the final veracity prediction
  • Figure 3: NLI model pipeline. $S_A$ refers to SentenceA and $S_B$ refers to SentneceB. $g_A$ refers to AMR graphA from $S_A$ and $g_B$ refers to AMR graphB from $S_B$
  • Figure 4: Abstract Meaning Representations (AMRs) for Claim: "Wish Upon was released in the 21st century." and Evidence: "It is set to be released in theaters on July 14, 2017, by Broad Green Pictures and Orion Pictures"
  • Figure 5: Abstract Meaning Representations (AMRs) for Claim: "Marnie is a romantic film." and Evidence: "Marnie is a 1964 American psychological thriller film directed by Alfred Hitchcock."
  • ...and 2 more figures