CAMRI Loss: Improving Recall of a Specific Class without Sacrificing Accuracy

TL;DR

The paper tackles the problem of improving recall for a single important class in multi-class classification without sacrificing overall accuracy, a critical concern in safety-sensitive tasks. It introduces CAMRI Loss, a class-sensitive additive angular margin that applies a margin only to the important class during training, enhancing intra-class compactness for that class. Across CIFAR-10, GTSRB, and AwA2, CAMRI achieves recall improvements up to 9% while maintaining accuracy, outperforming class-agnostic margins like ArcFace in recall. The approach effectively sharpens the separation between the important class and others by concentrating the margin where it matters, offering practical gains for real-world deployments with one or more high-risk classes.

Abstract

In real-world applications of multi-class classification models, misclassification in an important class (e.g., stop sign) can be significantly more harmful than in other classes (e.g., speed limit). In this paper, we propose a loss function that can improve the recall of an important class while maintaining the same level of accuracy as the case using cross-entropy loss. For our purpose, we need to make the separation of the important class better than the other classes. However, existing methods that give a class-sensitive penalty for cross-entropy loss do not improve the separation. On the other hand, the method that gives a margin to the angle between the feature vectors and the weight vectors of the last fully connected layer corresponding to each feature can improve the separation. Therefore, we propose a loss function that can improve the separation of the important class by setting the margin only for the important class, called Class-sensitive Additive Angular Margin Loss (CAMRI Loss). CAMRI loss is expected to reduce the variance of angles between features and weights of the important class relative to other classes due to the margin around the important class in the feature space by adding a penalty to the angle. In addition, concentrating the penalty only on the important classes hardly sacrifices the separation of the other classes. Experiments on CIFAR-10, GTSRB, and AwA2 showed that the proposed method could improve up to 9% recall improvement on cross-entropy loss without sacrificing accuracy.

Figures (4)

• Figure 1: The contour plots of the loss values formed by each loss function are shown. The vertical and horizontal axes represent the elements $z_1, z_2$ in the feature vector $\boldsymbol{z}$ and the contour plots represent the loss values, with yellow indicating high loss values and blue indicating low loss values. The weight vectors $\boldsymbol{W}_1, \boldsymbol{W}_2$ and $\boldsymbol{W}_3$ are elements in the last FC layer's weight vector. The ground truth weight vector $\boldsymbol{W}_1$ is shown in the red arrow. The non-ground truth weight vector $\boldsymbol{W}_2$ and $\boldsymbol{W}_3$ are shown in black arrows. The gray lines are the decision boundary.
• Figure 2: Normalized feature vectors of MNIST $\boldsymbol{z}^{\prime}$ (represented as dots) and weight vectors (represented as solid lines) in the two-dimensional feature space. The vertical and horizontal axes represent the elements $z_1^{\prime}, z_2^{\prime}$ of $\boldsymbol{z}^{\prime}$, respectively. The left shows L2-constrained softmax loss, the right shows ArcFace, and the bottom shows CAMRI loss. In the results, class "3" (shown in red) was set as the important class.
• Figure 3: Changes in recall of the important class (blue line) and accuracy (orange line) by varying penalties are shown. The left columns are WCE, and the right columns are CAMRI loss. From top to bottom, the important class is cat, dog, and airplane of CIFAR-10. The horizontal axis is the value of the penalty ($w_\kappa$ for WCE and $m_\kappa$ for CAMRI loss). The vertical axis is the value of recall of the important class and accuracy. The lines show the mean value in ten trials, and the bands show the standard deviation.
• Figure 4: The left (a) is the baseline confusion matrix of CIFAR-10 obtained by trained with cross-entropy loss. (b), (c), and (d) are the differences between the confusion matrix with cross-entropy loss (i.e., (a)) and that with the CAMRI loss, where cat, dog, and airplane be set to the important class, respectively. Each result is the average of ten trials and corresponds to the result shown in Table \ref{['tab:1']}.