The Multiclass Score-Oriented Loss (MultiSOL) on the Simplex

TL;DR

This work extends score-oriented losses from binary to multiclass classification by introducing a multidimensional threshold on the simplex. The MultiSOL framework defines a per-class confusion-matrix expectation under threshold distributions and optimizes a chosen score directly during training, using Monte Carlo sampling and differentiable surrogates. Empirical results across MNIST, FashionMNIST, CIFAR-10, and MedMNIST demonstrate robustness to priors and hyperparameters, competitive performance relative to state-of-the-art losses, and clear evidence of score-driven behavior. Overall, MultiSOL provides a principled mechanism to achieve metric-specific optimization in multiclass, particularly in imbalanced settings, linking simplex geometry with score-oriented learning.

Abstract

In the supervised binary classification setting, score-oriented losses have been introduced with the aim of optimizing a chosen performance metric directly during the training phase, thus avoiding \textit{a posteriori} threshold tuning. To do this, in their construction, the decision threshold is treated as a random variable provided with a certain \textit{a priori} distribution. In this paper, we use a recently introduced multidimensional threshold-based classification framework to extend such score-oriented losses to multiclass classification, defining the Multiclass Score-Oriented Loss (MultiSOL) functions. As also demonstrated by several classification experiments, this proposed family of losses is designed to preserve the main advantages observed in the binary setting, such as the direct optimization of the target metric and the robustness to class imbalance, achieving performance comparable to other state-of-the-art loss functions and providing new insights into the interaction between simplex geometry and score-oriented learning.

Figures (11)

• Figure 1: For $m=3$, the three regions $R_1(\boldsymbol{\tau})$ (red), $R_2(\boldsymbol{\tau})$ (green) and $R_3(\boldsymbol{\tau})$ (blue). From left to right: $\boldsymbol{\tau}=(1/3,1/3,1/3)$ (i.e., the argmax rule), $\boldsymbol{\tau}=(1/2,1/3,1/6)$, $\boldsymbol{\tau}=(1/8,3/4,1/8)$ (big black dot). The color blue, red or green represents the true label of the samples (colored dots). Number of misclassifications from left to right: $7$, $8$ and $10$ (from Legnaro25Multiclass).
• Figure 2: Heatmap of classification accuracies on the simplex (top row) and corresponding a priori Dirichlet distributions (bottom row) for varying parameters $\alpha = 5,10,20$.
• Figure 3: Class distribution of the MNIST dataset for the training and test splits. This shows the expected near-uniform distribution across all ten digits, with only minor fluctuations between classes.
• Figure 4: Class distribution of the BloodMNIST dataset across the training, validation, and test splits. Classes are basophil, eosinophil, erythroblast, immature granulocytes (myelocytes, metamyelocytes and promyelocytes), lymphocyte, monocyte, neutrophil, and platelet. The splits exhibit a consistent moderate imbalance, with neutrophils and eosinophils being most common and basophils and lymphocytes least represented.
• Figure 5: Class distribution of the DermaMNIST dataset across the training, validation, and test splits. The classes are: actinic keratoses and intraepithelial carcinoma, basal cell carcinoma, benign keratosis-like lesions, dermatofibroma, melanoma, melanocytic nevi, and vascular lesions. The dataset is highly imbalanced, dominated by melanocytic nevi (about $67\%$ in all splits), while all other classes occur at much lower frequencies.

Theorems & Definitions (6)

• Definition 1
• Definition 2
• Definition 3
• Definition 4
• Proposition 1
• Remark 1