Robust Semi-Supervised Learning for Self-learning Open-World Classes

TL;DR

Open-world SSL aims to classify known classes while discovering multiple unknown classes in unlabeled data. The paper introduces SSOC, which uses cross-attention between image features and learned class centers (prototypes) to explicitly model category concepts and self-learn multiple unknown classes, updating centers residually as $\mathcal{A}_{t+1}=\mathcal{A}_{t}+\Delta\mathcal{A}$. The learning objective combines a supervised term on labeled data, a pairwise similarity loss on sample pairs, and a maximum-entropy regularizer, with thresholds $\tau_1$ and $\tau_2$ guiding pseudo-labeling and pair selection; results on CIFAR-10/100 and ImageNet-100 show state-of-the-art performance, including strong gains in high novel-ratio scenarios (e.g., ImageNet-100 with 90% novel). The approach demonstrates robustness under limited labeled data and many novel classes, providing a practical, interpretable pathway for open-world recognition where unknown concepts must be discovered and distinguished.

Abstract

Existing semi-supervised learning (SSL) methods assume that labeled and unlabeled data share the same class space. However, in real-world applications, unlabeled data always contain classes not present in the labeled set, which may cause classification performance degradation of known classes. Therefore, open-world SSL approaches are researched to handle the presence of multiple unknown classes in the unlabeled data, which aims to accurately classify known classes while fine-grained distinguishing different unknown classes. To address this challenge, in this paper, we propose an open-world SSL method for Self-learning Open-world Classes (SSOC), which can explicitly self-learn multiple unknown classes. Specifically, SSOC first defines class center tokens for both known and unknown classes and autonomously learns token representations according to all samples with the cross-attention mechanism. To effectively discover novel classes, SSOC further designs a pairwise similarity loss in addition to the entropy loss, which can wisely exploit the information available in unlabeled data from instances' predictions and relationships. Extensive experiments demonstrate that SSOC outperforms the state-of-the-art baselines on multiple popular classification benchmarks. Specifically, on the ImageNet-100 dataset with a novel ratio of 90%, SSOC achieves a remarkable 22% improvement.

Figures (4)

• Figure 1: SSL settings do not include the concept of novel classes. In open-set SSL, the unknown classes in the unlabeled dataset are treated as a single category, and exclusively focus on known class classification. In open-world SSL, multiple unknown classes can be distinguished. Unknown class samples are highlighted with a red border.
• Figure 2: SSOC framework. SSOC models the concept of explicit classes and utilizes a cross-attention mechanism to facilitate dynamic interaction between image features and class centers. Through iterative steps, SSOC achieves self-learning of class centers. During inference, SSOC predicts the class based on the similarity between image features and class centers.
• Figure 3: Analysis of threshold effect on the CIFAR-100 dataset. Fig. \ref{['fig:para1']}(a) and Fig. \ref{['fig:para1']}(b) illustrate the number of selected pseudo-labels and the corresponding accuracy for different values of $\tau_1$. Fig. \ref{['fig:para1']}(c) presents the accuracy of the selected pseudo-labels for different values of $\tau_2$.
• Figure 4: Analysis of the effect of loss balance hyper-parameters on the ImageNet-100 dataset. We select two sets of parameter configurations, and in Fig. \ref{['fig:para2']}(a), Fig. \ref{['fig:para2']}(b), and Fig. \ref{['fig:para2']}(c), we present the accuracy on seen classes, unseen classes, and overall classes, respectively.