Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Ustnlp16 at SemEval-2025 Task 9: Improving Model Performance through Imbalance Handling and Focal Loss

Zhuoang Cai, Zhenghao Li, Yang Liu, Liyuan Guo, Yangqiu Song

TL;DR

This work addresses severe class imbalance in food hazard detection by evaluating data balancing techniques—oversampling, Easy Data Augmentation (EDA), and focal loss—using transformer backbones (BERT and RoBERTa). An integrated augmentation pipeline combines oversampling after tokenization with EDA applied to minority samples, while focal loss with tuned parameters emphasizes hard examples. Results show EDA notably improves accuracy and F1, and that combining EDA with oversampling and focal loss yields additional robustness for hard-to-classify hazards, with RoBERTa performing similarly to BERT in some setups. The findings offer practical guidance for imbalanced short-text classification in public-safety domains and underscore EDA as a strong single technique, with potential gains from complementary balancing methods.

Abstract

Classification tasks often suffer from imbal- anced data distribution, which presents chal- lenges in food hazard detection due to severe class imbalances, short and unstructured text, and overlapping semantic categories. In this paper, we present our system for SemEval- 2025 Task 9: Food Hazard Detection, which ad- dresses these issues by applying data augmenta- tion techniques to improve classification perfor- mance. We utilize transformer-based models, BERT and RoBERTa, as backbone classifiers and explore various data balancing strategies, including random oversampling, Easy Data Augmentation (EDA), and focal loss. Our ex- periments show that EDA effectively mitigates class imbalance, leading to significant improve- ments in accuracy and F1 scores. Furthermore, combining focal loss with oversampling and EDA further enhances model robustness, par- ticularly for hard-to-classify examples. These findings contribute to the development of more effective NLP-based classification models for food hazard detection.

Ustnlp16 at SemEval-2025 Task 9: Improving Model Performance through Imbalance Handling and Focal Loss

TL;DR

This work addresses severe class imbalance in food hazard detection by evaluating data balancing techniques—oversampling, Easy Data Augmentation (EDA), and focal loss—using transformer backbones (BERT and RoBERTa). An integrated augmentation pipeline combines oversampling after tokenization with EDA applied to minority samples, while focal loss with tuned parameters emphasizes hard examples. Results show EDA notably improves accuracy and F1, and that combining EDA with oversampling and focal loss yields additional robustness for hard-to-classify hazards, with RoBERTa performing similarly to BERT in some setups. The findings offer practical guidance for imbalanced short-text classification in public-safety domains and underscore EDA as a strong single technique, with potential gains from complementary balancing methods.

Abstract

Classification tasks often suffer from imbal- anced data distribution, which presents chal- lenges in food hazard detection due to severe class imbalances, short and unstructured text, and overlapping semantic categories. In this paper, we present our system for SemEval- 2025 Task 9: Food Hazard Detection, which ad- dresses these issues by applying data augmenta- tion techniques to improve classification perfor- mance. We utilize transformer-based models, BERT and RoBERTa, as backbone classifiers and explore various data balancing strategies, including random oversampling, Easy Data Augmentation (EDA), and focal loss. Our ex- periments show that EDA effectively mitigates class imbalance, leading to significant improve- ments in accuracy and F1 scores. Furthermore, combining focal loss with oversampling and EDA further enhances model robustness, par- ticularly for hard-to-classify examples. These findings contribute to the development of more effective NLP-based classification models for food hazard detection.
Paper Structure (20 sections, 2 equations, 1 figure, 4 tables)

This paper contains 20 sections, 2 equations, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Text Classification
  4. Data Augmentation
  5. Task Definition and Dataset
  6. Methodology
  7. Data Preprocessing and Augmentation
  8. Classification Model and Imbalance Handling
  9. System Configurations
  10. Experiments
  11. Experimental Setup
  12. Oversampling
  13. Easy Data Augmentation
  14. Focal Loss
  15. Training Details
  16. ...and 5 more sections

Figures (1)

  • Figure 1: The structured pipeline for food hazard detection.