A Survey of Deep Long-Tail Classification Advancements

TL;DR

This survey tackles deep long-tail classification by organizing algorithmic approaches into four interconnected branches: Loss Reweighting, Margin-Based Logit Adjustment, Optimized Representation Learning, and Balanced Classifier Learning. It furnishes a unified mathematical framework to compare methods, introduces comprehensive performance metrics beyond standard accuracy, and analyzes convergence and feature-distribution properties to diagnose LT behavior. Key findings include the strength of combining data- and algorithmic-level strategies, particularly data augmentation with supervised contrastive learning and logit-adjusted cross-entropy, as well as the complementary value of two-stage training and ensemble methods like RIDE/SADE. The study highlights persistent gaps in achieving uniformly high accuracy across many-shot, medium-shot, and few-shot regimes, especially under extreme or online imbalances, and argues for future work on zero-shot/online LT, robust regularization, and richer LT benchmarks. Overall, the framework serves as a practical guide for designing and evaluating LT classifiers and points to promising avenues where foundation models and self-supervised strategies may mitigate long-tail biases in real-world systems.

Abstract

Many data distributions in the real world are hardly uniform. Instead, skewed and long-tailed distributions of various kinds are commonly observed. This poses an interesting problem for machine learning, where most algorithms assume or work well with uniformly distributed data. The problem is further exacerbated by current state-of-the-art deep learning models requiring large volumes of training data. As such, learning from imbalanced data remains a challenging research problem and a problem that must be solved as we move towards more real-world applications of deep learning. In the context of class imbalance, state-of-the-art (SOTA) accuracies on standard benchmark datasets for classification typically fall less than 75%, even for less challenging datasets such as CIFAR100. Nonetheless, there has been progress in this niche area of deep learning. To this end, in this survey, we provide a taxonomy of various methods proposed for addressing the problem of long-tail classification, focusing on works that happened in the last few years under a single mathematical framework. We also discuss standard performance metrics, convergence studies, feature distribution and classifier analysis. We also provide a quantitative comparison of the performance of different SOTA methods and conclude the survey by discussing the remaining challenges and future research direction.

Figures (7)

• Figure 1: Denotes the drift of the decision boundary towards the minority class boundary with the increasing class imbalance ratio.
• Figure 2: Denotes the typical P(y) distributions in the real world data
• Figure 3: The figures indicate the class-wise hessian analysis of the loss for CIFAR 10 (IF=100:1) for CE and LDAM methods. The positive concentration of the eigenvalues for the loss function of the head class (0,1) indicates the convergence to the local minima. In contrast, the significant negative curvature for the tail classes(8,9) indicates the convergence of the convergence to saddle point.
• Figure 4: Logit margin distribution of the most frequent (class id-0) and the least frequent (class id-99) class for CIFAR 100 (imbalance factor =100:1) under CE-based model training with no long-tail adjustments.
• Figure 5: High-level categorization of algorithmic solutions for deep classification. Loss Rewighting* branch appears under Balanced Classifier Learning is redirected to Loss Reweighting main branch