Cardinality augmented loss functions
Miguel O'Malley
TL;DR
This work tackles class imbalance in neural networks by introducing cardinality augmented loss functions based on magnitude and spread, two metric space invariants that quantify effective diversity. Losses are applied batch-wise as L_orig with two divisive variants L_dmag and L_dspr to encourage diverse errors and balance training dynamics. Evaluations on synthetic imbalanced multiclass tasks and a real world DeepGlassNet dataset show that magnitude consistently improves minority-class performance and often enhances overall metrics, while spread provides more mixed results, with magnitude offering robust gains and manageable computation. The approach is simple to integrate as a drop-in loss replacement and opens avenues for further theoretical grounding and application to other domains and vector quantization.
Abstract
Class imbalance is a common and pernicious issue for the training of neural networks. Often, an imbalanced majority class can dominate training to skew classifier performance towards the majority outcome. To address this problem we introduce cardinality augmented loss functions, derived from cardinality-like invariants in modern mathematics literature such as magnitude and the spread. These invariants enrich the concept of cardinality by evaluating the `effective diversity' of a metric space, and as such represent a natural solution to overly homogeneous training data. In this work, we establish a methodology for applying cardinality augmented loss functions in the training of neural networks and report results on both artificially imbalanced datasets as well as a real-world imbalanced material science dataset. We observe significant performance improvement among minority classes, as well as improvement in overall performance metrics.