Boosting Consistency in Dual Training for Long-Tailed Semi-Supervised Learning

TL;DR

BOAT tackles long-tailed semi-supervised learning with unlabeled data from unknown distributions by employing a dual-branch framework: a standard branch that concentrates on head-class accuracy and a balanced branch that mitigates tail bias via logit adjustment. Through training-time interactions—alignment of the standard branch to a decoupled unlabeled distribution and the balanced branch to pseudo-labels derived from the standard branch, plus a post-hoc enhancement at inference—BOAT achieves robust performance across diverse unlabeled data distributions. Extensive experiments on CIFAR-10-LT, CIFAR-100-LT, STL10-LT, and ImageNet-127 show state-of-the-art gains, particularly under distribution mismatch, and ablations confirm the contributions of alignment, balanced pseudo-labels, and weighting. The work also demonstrates the potential of PEFT-based fine-tuning for further improvements, indicating practical relevance for real-world LTSSL tasks and future research directions.

Abstract

While long-tailed semi-supervised learning (LTSSL) has received tremendous attention in many real-world classification problems, existing LTSSL algorithms typically assume that the class distributions of labeled and unlabeled data are almost identical. Those LTSSL algorithms built upon the assumption can severely suffer when the class distributions of labeled and unlabeled data are mismatched since they utilize biased pseudo-labels from the model. To alleviate this problem, we propose a new simple method that can effectively utilize unlabeled data from unknown class distributions through Boosting cOnsistency in duAl Training (BOAT). Specifically, we construct the standard and balanced branch to ensure the performance of the head and tail classes, respectively. Throughout the training process, the two branches incrementally converge and interact with each other, eventually resulting in commendable performance across all classes. Despite its simplicity, we show that BOAT achieves state-of-the-art performance on a variety of standard LTSSL benchmarks, e.g., an averaged 2.7% absolute increase in test accuracy against existing algorithms when the class distributions of labeled and unlabeled data are mismatched. Even when the class distributions are identical, BOAT consistently outperforms many sophisticated LTSSL algorithms. We carry out extensive ablation studies to tease apart the factors that are the most important to the success of BOAT. The source code is available at https://github.com/Gank0078/BOAT.

Figures (9)

• Figure 1: \ref{['fig:dist_con', 'fig:dist_uni', 'fig:dist_rev']}): Three typical types of class distribution of unlabeled data: consistent, uniform, and reversed.
• Figure 2: More class distribution patterns for unlabeled data, i.e., middle and headtail.
• Figure 3: (\ref{['fig:pseu_acc']}): Pseudo-label accuracy for unlabeled data. The reported results are based on the average accuracy of consistent, uniform, and reversed with $\gamma_l=100$ on CIFAR-10-LT. (\ref{['fig:sen_tau', 'fig:sen_taus']}): The sensitivity of $\tau$ and $\tau_s$ under various settings on CIFAR-10-LT.
• Figure 4: More realistic LTSSL settings with various imbalance ratio for unlabeled data or labeled data. (\ref{['fig:more_cifar10']}): $N_1=500, M_1=4000$ with $\gamma_l=100$ on CIFAR-10-LT. (\ref{['fig:more_cifar100']}): $N_1=50, M_1=400$ with $\gamma_l=20$ on CIFAR-100-LT. (\ref{['fig:more_stl10']}): $N_1=150$ with various imbalanced ratio for labeled set on STL10-LT.
• Figure 5: (\ref{['fig:kl_qhatbal']}): The KL distance of predicted unlabeled data distribution between standard and balanced branch. The experiments are conducted on CIFAR-100-LT with $N_1=50, M_1=400,$ and $\gamma_l=10$. Con, Uni, and Rev represent consistent, uniform, and reversed for short. (\ref{['fig:cifar10_head', 'fig:cifar10_tail']}): The average precision and recall across different settings for head and tail classes. The experiments are conducted on CIFAR-10-LT with $N_1=500, M_1=4000,$ and $\gamma_l=100$. STD and BAL represent standard branch and balanced branch, respectively.