Trusted Multi-view Learning for Long-tailed Classification

TL;DR

This work tackles long-tailed classification in multi-view data by introducing TMLC, a framework that enforces trustworthy decisions through evidential deep learning and an oversampling-based strategy. It introduces Group Consensus Opinion Aggregation, inspired by Social Identity Theory, to bias decisions toward group consensus, and an uncertainty-guided, multi-view SMOTE-inspired pseudo-data generation driven by a distance metric on joint subjective evidence. The model employs an evidential learning paradigm with Dirichlet-distributed predictions and a loss that combines an adjusted cross-entropy term with a KL regularizer, forming L_acc = L_ace + lambda_t L_KL. Extensive experiments on HandWritten, PIE, Caltech101, NUS-WIDE, Scene15, and Animal demonstrate significant improvements over state-of-the-art baselines, especially for tail classes, underscoring the practical value of reliable multi-view long-tailed learning. The authors provide code to facilitate adoption of their approach in real-world scenarios.

Abstract

Class imbalance has been extensively studied in single-view scenarios; however, addressing this challenge in multi-view contexts remains an open problem, with even scarcer research focusing on trustworthy solutions. In this paper, we tackle a particularly challenging class imbalance problem in multi-view scenarios: long-tailed classification. We propose TMLC, a Trusted Multi-view Long-tailed Classification framework, which makes contributions on two critical aspects: opinion aggregation and pseudo-data generation. Specifically, inspired by Social Identity Theory, we design a group consensus opinion aggregation mechanism that guides decision making toward the direction favored by the majority of the group. In terms of pseudo-data generation, we introduce a novel distance metric to adapt SMOTE for multi-view scenarios and develop an uncertainty-guided data generation module that produces high-quality pseudo-data, effectively mitigating the adverse effects of class imbalance. Extensive experiments on long-tailed multi-view datasets demonstrate that our model is capable of achieving superior performance. The code is released at https://github.com/cncq-tang/TMLC.

Figures (6)

• Figure 1: Illustration of TMLC. Assuming that the opinions from the first $V-1$ views have been aggregated, we now focus on combining the resulting joint opinion $\boldsymbol{e}^{\Diamond(V-1)}$ with the opinion $\boldsymbol{e}^{V}$ from the $V$-th view. In this process, a larger weight is assigned to $\boldsymbol{e}^{\Diamond(V-1)}$ which respresents the group joint opinion (bold arrow line), while a smaller weight is assigned to the individual opinion $\boldsymbol{e}^{V}$ (thin arrow line). Subsequently, we propose an uncertainty-based sample generation scheme to ensure the production of reliable pseudo-data for the minority class.
• Figure 2: Density of overall uncertainty on the PIE dataset.
• Figure 3: Performance comparison on PIE datasets with varying decay rates.
• Figure 4: Accuracy comparison with respect to $R$.
• Figure 5: Joint evidence visualization on different datasets.