Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

REFLEX: Self-Refining Explainable Fact-Checking via Disentangling Truth into Style and Substance

Chuyi Kong, Gao Wei, Jing Ma, Hongzhan Lin, Yaxin Fan

TL;DR

REFLEX addresses the latency and reliability challenges of external-reliance AFC by exploiting internal backbone knowledge. It reformulates fact-checking as role-play with joint verdict and explanation training, then derives steering vectors from contrastive activation pairs to disentangle truth into substance and style. The approach achieves state-of-the-art verdict accuracy with a small set of self-refined samples and improves explanation quality, demonstrating strong transferability across backbones and pair configurations. Overall, REFLEX offers data-efficient, interpretable, and robust fact-checking with practical implications for real-time misinformation mitigation.

Abstract

The prevalence of misinformation on social media threatens public trust, demanding automated fact-checking systems that provide accurate verdicts with interpretable explanations. However, existing large language model-based (LLM-based) approaches often rely heavily on external knowledge sources, introducing substantial latency and even hallucinations that undermine reliability, interpretability, and responsiveness, which is crucial for real-time use. To address these challenges, we propose REason-guided Fact-checking with Latent EXplanations REFLEX paradigm, a plug-and-play, self-refining paradigm that leverages the internal knowledge in backbone model to improve both verdict accuracy and explanation quality. REFLEX reformulates fact-checking as a role-play dialogue and jointly trains verdict prediction and explanation generation. It adaptively extracts contrastive activation pairs between the backbone model and its fine-tuned variant to construct steering vectors that disentangle truth into style and substance naturally. These activation-level signals guide inference and suppress noisy explanations, enabling more faithful and efficient reasoning. Experiments on real-world datasets show that REFLEX outperforms previous methods that steer toward a single truth direction and underscores the challenge traditional approaches face when handling the subtle, human-unknown truth in fact-checking tasks. Remarkably, with only 465 self-refined training samples, RELFEX achieves state-of-the-art performance. Furthermore, models trained with explanatory objectives can effectively guide those without them, yielding up to a 7.57% improvement, highlighting that internal explanation signals play a dual role in both interpreting and enhancing factual reasoning.

REFLEX: Self-Refining Explainable Fact-Checking via Disentangling Truth into Style and Substance

TL;DR

REFLEX addresses the latency and reliability challenges of external-reliance AFC by exploiting internal backbone knowledge. It reformulates fact-checking as role-play with joint verdict and explanation training, then derives steering vectors from contrastive activation pairs to disentangle truth into substance and style. The approach achieves state-of-the-art verdict accuracy with a small set of self-refined samples and improves explanation quality, demonstrating strong transferability across backbones and pair configurations. Overall, REFLEX offers data-efficient, interpretable, and robust fact-checking with practical implications for real-time misinformation mitigation.

Abstract

The prevalence of misinformation on social media threatens public trust, demanding automated fact-checking systems that provide accurate verdicts with interpretable explanations. However, existing large language model-based (LLM-based) approaches often rely heavily on external knowledge sources, introducing substantial latency and even hallucinations that undermine reliability, interpretability, and responsiveness, which is crucial for real-time use. To address these challenges, we propose REason-guided Fact-checking with Latent EXplanations REFLEX paradigm, a plug-and-play, self-refining paradigm that leverages the internal knowledge in backbone model to improve both verdict accuracy and explanation quality. REFLEX reformulates fact-checking as a role-play dialogue and jointly trains verdict prediction and explanation generation. It adaptively extracts contrastive activation pairs between the backbone model and its fine-tuned variant to construct steering vectors that disentangle truth into style and substance naturally. These activation-level signals guide inference and suppress noisy explanations, enabling more faithful and efficient reasoning. Experiments on real-world datasets show that REFLEX outperforms previous methods that steer toward a single truth direction and underscores the challenge traditional approaches face when handling the subtle, human-unknown truth in fact-checking tasks. Remarkably, with only 465 self-refined training samples, RELFEX achieves state-of-the-art performance. Furthermore, models trained with explanatory objectives can effectively guide those without them, yielding up to a 7.57% improvement, highlighting that internal explanation signals play a dual role in both interpreting and enhancing factual reasoning.

Paper Structure

This paper contains 35 sections, 10 equations, 4 figures, 14 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Explainable Fact-Checking
  4. Style and Substance in Fact-Checking
  5. Methodology
  6. Task Formulation
  7. Model Training
  8. Data Preprocessing
  9. Training Protocol
  10. Contrastive Pairs Extraction
  11. Self-Knowledge Distillation
  12. Adaptive Sample Selection
  13. Explanation-Guided Steering
  14. Logistic-Probe Learning
  15. Explanation Refinement
  16. ...and 20 more sections

Figures (4)

  • Figure 1: The brief outline of our three-stage REFLEX paradigm. The red text denotes reasoning style learned from fine-tuning, and the blue text denotes factual knowledge stored in backbone models.
  • Figure 2: Optimal Layer for improving pairs across different layers. Square brackets denote optional components.
  • Figure 3: The correlation matrix between the explanation quality and fact-checking performance.
  • Figure 4: The Redundancy Noise Pattern in LLaMA2 on RAW-FC, layer 10 with IV, multiplier 1.5. Red tokens denote alignment with optimal vector direction; blue denotes opposite.