Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Partial Distribution Alignment via Adaptive Optimal Transport

Pei Yang, Qi Tan, Guihua Wen

TL;DR

The proposed adaptive optimal transport is instantiate the adaptive optimal transport in machine learning application to align source and target distributions partially and adaptively by respecting the ubiquity of noises, outliers, and distribution shifts in the data.

Abstract

To remedy the drawbacks of full-mass or fixed-mass constraints in classical optimal transport, we propose adaptive optimal transport which is distinctive from the classical optimal transport in its ability of adaptive-mass preserving. It aims to answer the mathematical problem of how to transport the probability mass adaptively between probability distributions, which is a fundamental topic in various areas of artificial intelligence. Adaptive optimal transport is able to transfer mass adaptively in the light of the intrinsic structure of the problem itself. The theoretical results shed light on the adaptive mechanism of mass transportation. Furthermore, we instantiate the adaptive optimal transport in machine learning application to align source and target distributions partially and adaptively by respecting the ubiquity of noises, outliers, and distribution shifts in the data. The experiment results on the domain adaptation benchmarks show that the proposed method significantly outperforms the state-of-the-art algorithms.

Partial Distribution Alignment via Adaptive Optimal Transport

TL;DR

The proposed adaptive optimal transport is instantiate the adaptive optimal transport in machine learning application to align source and target distributions partially and adaptively by respecting the ubiquity of noises, outliers, and distribution shifts in the data.

Abstract

To remedy the drawbacks of full-mass or fixed-mass constraints in classical optimal transport, we propose adaptive optimal transport which is distinctive from the classical optimal transport in its ability of adaptive-mass preserving. It aims to answer the mathematical problem of how to transport the probability mass adaptively between probability distributions, which is a fundamental topic in various areas of artificial intelligence. Adaptive optimal transport is able to transfer mass adaptively in the light of the intrinsic structure of the problem itself. The theoretical results shed light on the adaptive mechanism of mass transportation. Furthermore, we instantiate the adaptive optimal transport in machine learning application to align source and target distributions partially and adaptively by respecting the ubiquity of noises, outliers, and distribution shifts in the data. The experiment results on the domain adaptation benchmarks show that the proposed method significantly outperforms the state-of-the-art algorithms.

Paper Structure

This paper contains 18 sections, 3 theorems, 41 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Optimal Transport
  4. Machine Learning via Optimal Transport
  5. Adaptive Optimal Transport
  6. The Primary Problem
  7. Partial Distribution Alignment
  8. Theoretical Analysis
  9. AOT vs POT
  10. Adaptive Mass Transport
  11. Duality Theory
  12. Experiments
  13. Experiment Setup
  14. Adaptive Mass Transport
  15. Robustness of Partial Mapping
  16. ...and 3 more sections

Key Result

Theorem 1

Let $\gamma^*$ be the optimizer for the adaptive optimal transport problem defined in Equation pp. For the pair $(x,z)$ with positive cost, there is no mass transferred between them. For the pair $(x,z)$ with negative cost, either the mass taken from $x$ coincides with $d \mu(x)$, or the mass transf And it is not necessary that both equations hold.

Figures (5)

  • Figure 1: A toy example illustrating the partial distribution alignment via adaptive optimal transport. The line linked two samples represents the mass transport between them. The masses are transferred between the active regions $X^A \cup Z^A$, while there is no transportation of masses between inactive regions $X^I \cup Z^I$. Also, the samples connected with lines form the clusters in active regions, and the isolated samples in inactive regions are likely to be outliers or noises.
  • Figure 2: Heatmap of optimal transport plan illustrating the mass allocation mechanism of adaptive optimal transport. The $72 \times 72$ transport plan matrix is partitioned into $12 \times 12$ blocks. Most of the masses are allocated along the diagonal blocks, aligning labels between source domain and target domain.
  • Figure 3: Heatmap of label-wise transport plan in the case of partial mapping (top panels). The masses are aggregated by the labels. The histograms plot the label-wise marginal distributions for the source and target domains respectively (bottom panels).
  • Figure 4: Sensitivity analysis on $\beta$.
  • Figure 5: Sensitivity analysis on $\epsilon$.

Theorems & Definitions (5)

  • Theorem 1: Optimal Transport Mass
  • proof
  • Theorem 2: Active Regions vs. Inactive Regions
  • Theorem 3: Duality of Adaptive Optimal Transport
  • proof