Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple Logits Retargeting Approach

Han Lu, Siyu Sun, Yichen Xie, Liqing Zhang, Xiaokang Yang, Junchi Yan

TL;DR

This work rethinks classifier re-training in long-tailed recognition by evaluating methods under a unified feature representation and introducing Logits Magnitude and Regularized Standard Deviation as informative metrics. It argues that direct optimization of logits magnitude is challenging, and proposes Logits Retargeting (LORT), a simple, class-count-agnostic retraining strategy that reshapes labels so each class receives substantial negative probability mass, effectively shrinking Logits Magnitude. Empirically, LORT achieves state-of-the-art performance on CIFAR100-LT, ImageNet-LT, and iNaturalist2018 and demonstrates robustness to hyperparameters, while acting as a plug-and-play improvement for existing models. The practical impact is a simple, effective, and broadly applicable method for boosting minority-class performance in imbalanced recognition tasks, along with two metrics to better analyze and compare retraining strategies.

Abstract

In the long-tailed recognition field, the Decoupled Training paradigm has demonstrated remarkable capabilities among various methods. This paradigm decouples the training process into separate representation learning and classifier re-training. Previous works have attempted to improve both stages simultaneously, making it difficult to isolate the effect of classifier re-training. Furthermore, recent empirical studies have demonstrated that simple regularization can yield strong feature representations, emphasizing the need to reassess existing classifier re-training methods. In this study, we revisit classifier re-training methods based on a unified feature representation and re-evaluate their performances. We propose a new metric called Logits Magnitude as a superior measure of model performance, replacing the commonly used Weight Norm. However, since it is hard to directly optimize the new metric during training, we introduce a suitable approximate invariant called Regularized Standard Deviation. Based on the two newly proposed metrics, we prove that reducing the absolute value of Logits Magnitude when it is nearly balanced can effectively decrease errors and disturbances during training, leading to better model performance. Motivated by these findings, we develop a simple logits retargeting approach (LORT) without the requirement of prior knowledge of the number of samples per class. LORT divides the original one-hot label into small true label probabilities and large negative label probabilities distributed across each class. Our method achieves state-of-the-art performance on various imbalanced datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist2018.

Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple Logits Retargeting Approach

TL;DR

This work rethinks classifier re-training in long-tailed recognition by evaluating methods under a unified feature representation and introducing Logits Magnitude and Regularized Standard Deviation as informative metrics. It argues that direct optimization of logits magnitude is challenging, and proposes Logits Retargeting (LORT), a simple, class-count-agnostic retraining strategy that reshapes labels so each class receives substantial negative probability mass, effectively shrinking Logits Magnitude. Empirically, LORT achieves state-of-the-art performance on CIFAR100-LT, ImageNet-LT, and iNaturalist2018 and demonstrates robustness to hyperparameters, while acting as a plug-and-play improvement for existing models. The practical impact is a simple, effective, and broadly applicable method for boosting minority-class performance in imbalanced recognition tasks, along with two metrics to better analyze and compare retraining strategies.

Abstract

In the long-tailed recognition field, the Decoupled Training paradigm has demonstrated remarkable capabilities among various methods. This paradigm decouples the training process into separate representation learning and classifier re-training. Previous works have attempted to improve both stages simultaneously, making it difficult to isolate the effect of classifier re-training. Furthermore, recent empirical studies have demonstrated that simple regularization can yield strong feature representations, emphasizing the need to reassess existing classifier re-training methods. In this study, we revisit classifier re-training methods based on a unified feature representation and re-evaluate their performances. We propose a new metric called Logits Magnitude as a superior measure of model performance, replacing the commonly used Weight Norm. However, since it is hard to directly optimize the new metric during training, we introduce a suitable approximate invariant called Regularized Standard Deviation. Based on the two newly proposed metrics, we prove that reducing the absolute value of Logits Magnitude when it is nearly balanced can effectively decrease errors and disturbances during training, leading to better model performance. Motivated by these findings, we develop a simple logits retargeting approach (LORT) without the requirement of prior knowledge of the number of samples per class. LORT divides the original one-hot label into small true label probabilities and large negative label probabilities distributed across each class. Our method achieves state-of-the-art performance on various imbalanced datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist2018.
Paper Structure (15 sections, 3 theorems, 18 equations, 5 figures, 3 tables)

This paper contains 15 sections, 3 theorems, 18 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Rethinking Classifier Re-Training
  4. Revisiting Classifier Re-Training methods
  5. Logits Magnitude
  6. Method
  7. Deep Dive into Logits Magnitude
  8. Logits Retargeting
  9. Experiments
  10. Datasets and Implementation Details
  11. Benchmark Results
  12. Ablation Study
  13. Conclusion
  14. Proof of Proposition 1 Bias Convergence
  15. Datasets and Implementation Details

Key Result

Proposition 1

Given a deterministic matrix $\mathbf{W}$, loss function $\mathcal{L}$ is a convex function with respect to $\mathbf{b}$ and will reach the global minimum as the corresponding bias $\mathbf{b}$ converges.

Figures (5)

  • Figure 1: An Intuitive View of Logits Magnitude Influence. The two models are trained on CIFAR100-LT with IR=100. We take the instance indexed $7202$ in test set as an example. Its true class is $75$, but it is erroneously classified as class $3$ in Many class by Vanilla CE . The blue points represent the logits of the true positive class, while the red points correspond to the logits of the negative class. Our proposed method, LORT, can obtain the more balanced and smaller logits magnitudes (defined as the difference between the positive logits and negative logits of the same class), helping discrimination in a multi-class classification scenario.
  • Figure 2: Visual interpretation of Logits Magnitude. The Logits Magnitude L is defined as the difference between the mean logits of positive (dashed blue line) and negative (dashed red line) samples.
  • Figure 3: Overview of our proposed metrics. Left: Overview of Logits Magnitude for various methods on CIFAR100-LT with an imbalanced ratio of 100. Classes are grouped into segments of 10, and mean values are computed for comparative analysis. The methods in the legend are sorted in ascending order of performance from top to bottom. The difference in means between positive and negative samples is evaluated for each class in the test set. Magnitude regularization using the 1-norm is employed to enhance comparability. Right:Overview of Regularized Standard Deviation. The true distribution of logits $\mathbf{z}$ for each class is not available considering the lack of the samples, so the displayed results represent computations on the test set.
  • Figure 4: Effect of Label Smooth Value. Tuning smooth value can significantly influence the performance. The overall accuracy will be improved with the growth of smooth value. Emphasis on negative samples probability can effectively enhance the performance of the minority while also improving overall performance.
  • Figure 5: Sensitivity Comparison of Learning Rate and Weight Decay. Tuning learning rate (LR) and weight decay (WD) under the same finetuning epoch budget can have a significant impact on accuracy. Our method exhibits the strong stability across different LR and WD combinations.

Theorems & Definitions (7)

  • Proposition 1: Bias Convergence
  • proof
  • Definition 1: Logits Magnitude
  • Proposition 2: Arbitrary Vector Length
  • Definition 2: Regularized Standard Deviation
  • Proposition 3
  • proof