NCL++: Nested Collaborative Learning for Long-Tailed Visual Recognition

TL;DR

This work tackles long-tailed visual recognition by addressing prediction uncertainty across multiple experts and augmented copies. It introduces Nested Collaborative Learning (NCL++), combining Balanced Individual Learning with inter- and intra-expert online distillation, plus Hard Category Mining and Nested Feature Learning to emphasize hard negatives while maintaining global discrimination. The approach yields state-of-the-art results on five major long-tailed benchmarks and offers strong tail-class improvements with a single-network inference option, aided by careful ablations and analyses. Overall, NCL++ advances robust, fine-grained discrimination in imbalanced visual tasks with a balanced, nested distillation framework.

Abstract

Long-tailed visual recognition has received increasing attention in recent years. Due to the extremely imbalanced data distribution in long-tailed learning, the learning process shows great uncertainties. For example, the predictions of different experts on the same image vary remarkably despite the same training settings. To alleviate the uncertainty, we propose a Nested Collaborative Learning (NCL++) which tackles the long-tailed learning problem by a collaborative learning. To be specific, the collaborative learning consists of two folds, namely inter-expert collaborative learning (InterCL) and intra-expert collaborative learning (IntraCL). In-terCL learns multiple experts collaboratively and concurrently, aiming to transfer the knowledge among different experts. IntraCL is similar to InterCL, but it aims to conduct the collaborative learning on multiple augmented copies of the same image within the single expert. To achieve the collaborative learning in long-tailed learning, the balanced online distillation is proposed to force the consistent predictions among different experts and augmented copies, which reduces the learning uncertainties. Moreover, in order to improve the meticulous distinguishing ability on the confusing categories, we further propose a Hard Category Mining (HCM), which selects the negative categories with high predicted scores as the hard categories. Then, the collaborative learning is formulated in a nested way, in which the learning is conducted on not just all categories from a full perspective but some hard categories from a partial perspective. Extensive experiments manifest the superiority of our method with outperforming the state-of-the-art whether with using a single model or an ensemble. The code will be publicly released.

Figures (12)

• Figure 1: An illustration of long-tailed data distribution. In the distribution, few head classes occupy most of the data while many tail classes have only a few samples.
• Figure 2: (a) The Kullback–Leibler (KL) distance calculated from two aspects. The green color indicates the KL distance of two experts with taking the same image as the input. The blue color indicates the KL distance of an expert with taking two augmented image copies with respect to the same image as the inputs. (b) An illustration of the predictions produced by two experts with respect to the same input. The two experts have the same structure and are trained with the same settings. The analysis is conducted on CIFAR100-LT dataset with an Imbalanced Factor (IF) of 100. The predictions are visualized on the basis of a random selected example, and the KL distance is computed based on the whole test set and then the average results of each category are counted and reported. The predictions differ largely from each other between different networks and different augmented images. Bested viewed in color.
• Figure 3: An illustration of our proposed NCL++ of containing three experts and three augmented image copies for conducting collaborative learning. The proposed NCL++ contains five core components, namely Balanced Individual Learning (BIL), Intra-expert Collaborative Learning (IntraCL), Inter-expert Collaborative Learning (InterCL), Hard Category Mining (HCM) and Nested Feature Learning. The BIL aims to enhance discriminative ability of a single expert. Both IntraCL and InterCL aim to reduce the learning uncertainty by collaborative learning and thus improve the discriminative capability. For the proposed HCM, it selects the hard negative categories, which are used for improving the meticulous distinguishing ability of the model. Based on the selected hard categories, the NFL is further employed to learning features from a global view on all categories and also a partial view on hard categories.
• Figure 4: An illustration of the proposed HCM module.
• Figure 5: Parameter analysis of (a) the ratio $\beta$ and (b) the loss weight $\lambda$ on CIFAR100-LT dataset with IF of 100.