CLOC: Contrastive Learning for Ordinal Classification with Multi-Margin N-pair Loss

TL;DR

The paper tackles ordinal classification where misclassifications across adjacent ranks incur unequal costs and proposes CLOC, a margin-based contrastive learning framework that learns an ordered representation by optimizing multiple margins $\{m_h\}_{h\in\mathcal{H}}$ through the multi-margin N-Pair loss $\ell_{MM}$. By defining cumulative margins $\mathsf{m}_{y,y_k}$ and using a cosine similarity objective $\psi(\cdot,\cdot)$, CLOC enforces order while allowing flexible boundaries between adjacent ranks. The approach is trained in two phases to prevent margin collapse, and experiments on five real-world datasets plus a synthetic bias scenario show that CLOC achieves superior accuracy and provides controllability over critical decision boundaries and interpretability of learned margins. This margin-based, order-preserving framework has practical impact for high-stakes tasks such as medical image analysis, where different boundary decisions carry different consequences.

Abstract

In ordinal classification, misclassifying neighboring ranks is common, yet the consequences of these errors are not the same. For example, misclassifying benign tumor categories is less consequential, compared to an error at the pre-cancerous to cancerous threshold, which could profoundly influence treatment choices. Despite this, existing ordinal classification methods do not account for the varying importance of these margins, treating all neighboring classes as equally significant. To address this limitation, we propose CLOC, a new margin-based contrastive learning method for ordinal classification that learns an ordered representation based on the optimization of multiple margins with a novel multi-margin n-pair loss (MMNP). CLOC enables flexible decision boundaries across key adjacent categories, facilitating smooth transitions between classes and reducing the risk of overfitting to biases present in the training data. We provide empirical discussion regarding the properties of MMNP and show experimental results on five real-world image datasets (Adience, Historical Colour Image Dating, Knee Osteoarthritis, Indian Diabetic Retinopathy Image, and Breast Carcinoma Subtyping) and one synthetic dataset simulating clinical decision bias. Our results demonstrate that CLOC outperforms existing ordinal classification methods and show the interpretability and controllability of CLOC in learning meaningful, ordered representations that align with clinical and practical needs.

Figures (9)

• Figure 1: An illustration of the impact of classification errors in ordinal contexts. Standard classification treats all errors equally, while ordinal classification penalizes larger rank errors more heavily. CLOC further accounts for the varying impact of adjacent errors, particularly across critical boundaries like benign versus malignant categories or minor vs threshold age of majority. This figure shows samples from IDRID (cancer grading) ds_idrid and Adience (age estimation) ds_adience datasets. Longer arrow indicates a greater impact from adjacent errors.
• Figure 2: Set of data samples from different classes/ranks (indicated by colors), with their rank order relative to the anchor $z_1$ and its positive sample $z^+$, where $\text{rank } 1 \prec \text{rank } 2 \prec \text{rank } 3 \prec \text{rank } 4$. The MMNP loss pulls samples of the same rank closer to the anchor while pushing those with larger rank differences farther apart, resulting in wider margins in the representation space. Additionally, we can control specific margin values (e.g., by applying constraints) and use the learned margins to interpret class boundaries.
• Figure 3: Two phase training for CLOC. In the first phase, the encoder, classifier and the margins are trained with the multi-margin contrastive loss and classification loss. In the second phase, the margins are frozen allowing the training of the encoder and classifier with the losses.
• Figure 4: Training phase one with and without margin collapse "precaution" measures. Notice that training continues for larger number of epochs without margin collapse when "precaution" measures are applied.
• Figure 5: UMAP visualizations of learned representations for the IDRID dataset that focuses on cancer grade classification. Notice that proposed CLOC, effectively captures the inherent ordinal structure of cancer grades and maintains a larger margin at critical decision boundary, compared to other decision boundaries. Additional visualizations of GOL gol and POE poe are in Appendix \ref{['sec:more_visuals']}.