Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Synthetic Information towards Maximum Posterior Ratio for deep learning on Imbalanced Data

Hung Nguyen, Morris Chang

TL;DR

This paper tackles the challenge of class imbalance in deep learning, especially on tabular data, by proposing SIMPOR, a data-centric method that prioritizes informative minority samples through entropy-based active learning and guides synthetic data generation with a maximum posterior-ratio objective. Synthetic minority data are created on spheres around minority samples, with radii drawn from a Gaussian distribution and likelihoods estimated via kernel density estimation to compute a posterior ratio that steers samples toward the minority region while preserving data topology. The approach is evaluated on 41 real datasets and a Moon dataset, showing superior F1-score and AUC performance compared to SMOTE variants, GDO, and DeepSMOTE, with Wilcoxon tests confirming statistical significance. Although SIMPOR incurs higher preprocessing time due to KDE, its accuracy gains and principled handling of topology and informative regions argue for its utility in imbalanced deep learning contexts, with future work extending to image data and further parameter analyses.

Abstract

This study examines the impact of class-imbalanced data on deep learning models and proposes a technique for data balancing by generating synthetic data for the minority class. Unlike random-based oversampling, our method prioritizes balancing the informative regions by identifying high entropy samples. Generating well-placed synthetic data can enhance machine learning algorithms accuracy and efficiency, whereas poorly-placed ones may lead to higher misclassification rates. We introduce an algorithm that maximizes the probability of generating a synthetic sample in the correct region of its class by optimizing the class posterior ratio. Additionally, to maintain data topology, synthetic data are generated within each minority sample's neighborhood. Our experimental results on forty-one datasets demonstrate the superior performance of our technique in enhancing deep-learning models.

Synthetic Information towards Maximum Posterior Ratio for deep learning on Imbalanced Data

TL;DR

This paper tackles the challenge of class imbalance in deep learning, especially on tabular data, by proposing SIMPOR, a data-centric method that prioritizes informative minority samples through entropy-based active learning and guides synthetic data generation with a maximum posterior-ratio objective. Synthetic minority data are created on spheres around minority samples, with radii drawn from a Gaussian distribution and likelihoods estimated via kernel density estimation to compute a posterior ratio that steers samples toward the minority region while preserving data topology. The approach is evaluated on 41 real datasets and a Moon dataset, showing superior F1-score and AUC performance compared to SMOTE variants, GDO, and DeepSMOTE, with Wilcoxon tests confirming statistical significance. Although SIMPOR incurs higher preprocessing time due to KDE, its accuracy gains and principled handling of topology and informative regions argue for its utility in imbalanced deep learning contexts, with future work extending to image data and further parameter analyses.

Abstract

This study examines the impact of class-imbalanced data on deep learning models and proposes a technique for data balancing by generating synthetic data for the minority class. Unlike random-based oversampling, our method prioritizes balancing the informative regions by identifying high entropy samples. Generating well-placed synthetic data can enhance machine learning algorithms accuracy and efficiency, whereas poorly-placed ones may lead to higher misclassification rates. We introduce an algorithm that maximizes the probability of generating a synthetic sample in the correct region of its class by optimizing the class posterior ratio. Additionally, to maintain data topology, synthetic data are generated within each minority sample's neighborhood. Our experimental results on forty-one datasets demonstrate the superior performance of our technique in enhancing deep-learning models.
Paper Structure (33 sections, 9 equations, 8 figures, 9 tables, 2 algorithms)

This paper contains 33 sections, 9 equations, 8 figures, 9 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Sampling-based approach.
  4. Cost-sensitive learning approach.
  5. Ensemple learning approach.
  6. Preliminaries
  7. Imbalance Ratio (IR)
  8. Evaluation Metrics
  9. Entropy-based Active Learning
  10. The Problem of Learning From Imbalanced Datasets
  11. Methodology
  12. Methodology Motivation
  13. Generating minority synthetic data
  14. Algorithm
  15. Algorithm Time Complexity.
  16. ...and 18 more sections

Figures (8)

  • Figure 1: Learning from imbalanced datasets
  • Figure 2: Demonstration on how SIMPOR generates three synthetic samples $x'_0, x'_1, x'_2$, from three minority samples $x_0, x_1, x_2$, by maximizing the Posterior Ratio.
  • Figure 3: Artificial class imbalanced Moon dataset with IR of 7:1.
  • Figure 4: Data plot and model's decision boundary visualization for Moon Dataset over different techniques.
  • Figure 5: Winning times over 41 datasets.
  • ...and 3 more figures