Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Loss Functions for Fact Verification

Yuta Mukobara, Yutaro Shigeto, Masashi Shimbo

TL;DR

Two task-specific objectives tailored to FEVER are developed that outperform the standard cross-entropy loss and are combined with simple class weighting, which effectively overcomes the imbalance in the training data.

Abstract

We explore loss functions for fact verification in the FEVER shared task. While the cross-entropy loss is a standard objective for training verdict predictors, it fails to capture the heterogeneity among the FEVER verdict classes. In this paper, we develop two task-specific objectives tailored to FEVER. Experimental results confirm that the proposed objective functions outperform the standard cross-entropy. Performance is further improved when these objectives are combined with simple class weighting, which effectively overcomes the imbalance in the training data. The souce code is available at https://github.com/yuta-mukobara/RLF-KGAT

Rethinking Loss Functions for Fact Verification

TL;DR

Two task-specific objectives tailored to FEVER are developed that outperform the standard cross-entropy loss and are combined with simple class weighting, which effectively overcomes the imbalance in the training data.

Abstract

We explore loss functions for fact verification in the FEVER shared task. While the cross-entropy loss is a standard objective for training verdict predictors, it fails to capture the heterogeneity among the FEVER verdict classes. In this paper, we develop two task-specific objectives tailored to FEVER. Experimental results confirm that the proposed objective functions outperform the standard cross-entropy. Performance is further improved when these objectives are combined with simple class weighting, which effectively overcomes the imbalance in the training data. The souce code is available at https://github.com/yuta-mukobara/RLF-KGAT
Paper Structure (28 sections, 20 equations, 28 tables)

This paper contains 28 sections, 20 equations, 28 tables.

Table of Contents

  1. Introduction
  2. Notation
  3. Proposed Method
  4. Cross-entropy Loss Function
  5. Loss Functions for Verdict Prediction
  6. Multi-label logistic loss
  7. Application to verdict prediction
  8. Reducing penalties for false NEI
  9. Exclusive penalties for ${\textsf{SUP}}$/${\textsf{REF}}$ confusion
  10. Class Imbalanced Learning
  11. Experiments
  12. Setups
  13. Dataset and evaluation criteria
  14. Compared models and hyperparameters
  15. Results
  16. ...and 13 more sections