Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

P$^2$OT: Progressive Partial Optimal Transport for Deep Imbalanced Clustering

Chuyu Zhang, Hui Ren, Xuming He

TL;DR

This work tackles deep clustering under realistic class-imbalance (deep imbalanced clustering) by marrying progressive pseudo-labeling with a Progressive Partial Optimal Transport (P$^2$OT) framework. P$^2$OT imposes imbalance-aware pseudo-labels through a KL-divergence cluster-size constraint and a total-mass constraint that progressively selects samples, reformulated as an unbalanced OT problem and solved via a fast Sinkhorn-like scaling method with a virtual cluster. The method enables one-stage end-to-end learning, exploiting a memory buffer and an adaptive mass ramp to learn from easy to hard samples without hand-tuned confidence thresholds. Experiments on CIFAR100 with long-tail, ImageNet-R, and large iNaturalist subsets show state-of-the-art performance, particularly improving medium and tail classes while maintaining efficiency on large-scale data.

Abstract

Deep clustering, which learns representation and semantic clustering without labels information, poses a great challenge for deep learning-based approaches. Despite significant progress in recent years, most existing methods focus on uniformly distributed datasets, significantly limiting the practical applicability of their methods. In this paper, we first introduce a more practical problem setting named deep imbalanced clustering, where the underlying classes exhibit an imbalance distribution. To tackle this problem, we propose a novel pseudo-labeling-based learning framework. Our framework formulates pseudo-label generation as a progressive partial optimal transport problem, which progressively transports each sample to imbalanced clusters under prior distribution constraints, thus generating imbalance-aware pseudo-labels and learning from high-confident samples. In addition, we transform the initial formulation into an unbalanced optimal transport problem with augmented constraints, which can be solved efficiently by a fast matrix scaling algorithm. Experiments on various datasets, including a human-curated long-tailed CIFAR100, challenging ImageNet-R, and large-scale subsets of fine-grained iNaturalist2018 datasets, demonstrate the superiority of our method.

P$^2$OT: Progressive Partial Optimal Transport for Deep Imbalanced Clustering

TL;DR

This work tackles deep clustering under realistic class-imbalance (deep imbalanced clustering) by marrying progressive pseudo-labeling with a Progressive Partial Optimal Transport (POT) framework. POT imposes imbalance-aware pseudo-labels through a KL-divergence cluster-size constraint and a total-mass constraint that progressively selects samples, reformulated as an unbalanced OT problem and solved via a fast Sinkhorn-like scaling method with a virtual cluster. The method enables one-stage end-to-end learning, exploiting a memory buffer and an adaptive mass ramp to learn from easy to hard samples without hand-tuned confidence thresholds. Experiments on CIFAR100 with long-tail, ImageNet-R, and large iNaturalist subsets show state-of-the-art performance, particularly improving medium and tail classes while maintaining efficiency on large-scale data.

Abstract

Deep clustering, which learns representation and semantic clustering without labels information, poses a great challenge for deep learning-based approaches. Despite significant progress in recent years, most existing methods focus on uniformly distributed datasets, significantly limiting the practical applicability of their methods. In this paper, we first introduce a more practical problem setting named deep imbalanced clustering, where the underlying classes exhibit an imbalance distribution. To tackle this problem, we propose a novel pseudo-labeling-based learning framework. Our framework formulates pseudo-label generation as a progressive partial optimal transport problem, which progressively transports each sample to imbalanced clusters under prior distribution constraints, thus generating imbalance-aware pseudo-labels and learning from high-confident samples. In addition, we transform the initial formulation into an unbalanced optimal transport problem with augmented constraints, which can be solved efficiently by a fast matrix scaling algorithm. Experiments on various datasets, including a human-curated long-tailed CIFAR100, challenging ImageNet-R, and large-scale subsets of fine-grained iNaturalist2018 datasets, demonstrate the superiority of our method.
Paper Structure (38 sections, 2 theorems, 35 equations, 7 figures, 11 tables, 3 algorithms)

This paper contains 38 sections, 2 theorems, 35 equations, 7 figures, 11 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminary
  4. Method
  5. Problem Setup and Method Overview
  6. Progressive Partial Optimal Transport (P$^2$OT)
  7. Increasing strategy of $\rho$.
  8. Efficient Solver for P$^2$OT
  9. Experiments
  10. Experimental Setup
  11. Datasets.
  12. Evaluation Metric.
  13. Implementation Details.
  14. Main Results
  15. Ablation Study
  16. ...and 23 more sections

Key Result

Proposition 1

If $\mathbf C = [-\log \mathbf P, \bm 0_N]$, and $\bm\lambda_{:K}=\lambda, \bm\lambda_{K+1} \rightarrow +\infty$, the optimal transport plan $\hat{\mathbf{Q}}^\star$ of Equ.(eq:re_curr_unbalanced_ot_2) can be expressed as: where $\mathbf{Q}^\star$ is optimal transport plan of Equ.(eq:curr_unbalanced_ot), and $\bm\xi^\star$ is the last column of $\hat{\mathbf{Q}}^\star$.

Figures (7)

  • Figure 1: Head, Medium, and Tail comparison on several datasets.
  • Figure 2: The T-SNE analysis on iNature100 training set.
  • Figure 3: Confusion matrix on the balanced CIFAR100 test set. The two red rectangles represent the Medium and Tail classes.
  • Figure 4: Precision, Recall analysis on train dataset with different training epoch. The weighted precision and recall are derived by reweighting each sample by our P$^2$OT algorithm.
  • Figure 5: Time cost comparison of our solver with Generalized Scaling Algorithm (GSA).
  • ...and 2 more figures

Theorems & Definitions (2)

  • Proposition 1
  • Proposition 2