Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sinkhorn algorithms for entropic vector quantile regression

Kengo Kato, Boyu Wang

Abstract

Vector quantile regression (VQR) is an optimal transport (OT)-based framework that extends linear quantile regression to vector-valued response variables and can be formulated as an OT problem with a mean-independence constraint. In this paper, we study two Sinkhorn-type algorithms for VQR with entropic regularization, building on our previous work on its duality theory. The first is a direct adaptation of the classical Sinkhorn iteration based on solving the full Schrödinger-type system characterizing the dual potentials, which requires solving an implicit functional equation at each iteration. The second algorithm, which is new in the literature, replaces the implicit update with a projected gradient step, resulting in a modified scheme that is computationally more practical. For both algorithms, and for general compactly supported marginals, we establish linear convergence in both the dual objective value and the iterates. A key innovation in our analysis is the derivation of explicit quantitative bounds on the dual potentials and Sinkhorn iterates.

Sinkhorn algorithms for entropic vector quantile regression

Abstract

Vector quantile regression (VQR) is an optimal transport (OT)-based framework that extends linear quantile regression to vector-valued response variables and can be formulated as an OT problem with a mean-independence constraint. In this paper, we study two Sinkhorn-type algorithms for VQR with entropic regularization, building on our previous work on its duality theory. The first is a direct adaptation of the classical Sinkhorn iteration based on solving the full Schrödinger-type system characterizing the dual potentials, which requires solving an implicit functional equation at each iteration. The second algorithm, which is new in the literature, replaces the implicit update with a projected gradient step, resulting in a modified scheme that is computationally more practical. For both algorithms, and for general compactly supported marginals, we establish linear convergence in both the dual objective value and the iterates. A key innovation in our analysis is the derivation of explicit quantitative bounds on the dual potentials and Sinkhorn iterates.
Paper Structure (20 sections, 15 theorems, 165 equations, 2 figures, 2 algorithms)

This paper contains 20 sections, 15 theorems, 165 equations, 2 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Overview
  3. Related literature
  4. Organization
  5. Notation
  6. Duality for entropic VQR
  7. Sinkhorn algorithm
  8. Modified Sinkhorn algorithm
  9. Numerical experiments
  10. Synthetic data: Gaussian case
  11. Real data: iris dataset
  12. Results
  13. Proofs for Sections \ref{['sec: duality']} and \ref{['sec: sinkhorn']}
  14. Proof of Proposition \ref{['prop: potential bound']}
  15. Proof of Proposition \ref{['prop: sinkhorn potential']}
  16. ...and 5 more sections

Key Result

Proposition 2.1

Let $(\bar{f},\bar{g},\bar{h}) \in \mathcal{C}_\diamond (\mathcal{U}) \times \mathcal{C}_\diamond (\mathcal{U};\mathbb{R}^{d_x}) \times \mathcal{C}(\mathcal{X} \times \mathcal{Y})$ be dual potentials. Then we have

Figures (2)

  • Figure 1: Log duality gaps for the Gaussian setting. The gap represents $D(\bar{f},\bar{g},\bar{h})-D(\widehat{f}^t,\widehat{g}^t,\widehat{h}^t)$, where the closed-form expression in (\ref{['eq: gaussian dual value']}) is used for $D(\bar{f},\bar{g},\bar{h})$.
  • Figure 2: Log duality gaps for the iris dataset. The gap represents $D(\widehat{f}^{t_{\max}},\widehat{g}^{t_{\max}},\widehat{h}^{t_{\max}})-D(\widehat{f}^t,\widehat{g}^t,\widehat{h}^t)$ with $t_{\max}=100$.

Theorems & Definitions (35)

  • Remark 2.1: Conventions on dual potentials
  • Proposition 2.1: Quantitative upper bounds on dual potentials
  • Definition 3.1: Sinkhorn algorithm for entropic VQR
  • Remark 3.1
  • Proposition 3.1: Quantitative upper bounds on Sinkhorn iterates
  • Theorem 3.1: Linear convergence of Sinkhorn algorithm
  • Definition 4.1: Modified Sinkhorn algorithm for entropic VQR
  • Remark 4.1
  • Theorem 4.1: Linear convergence of modified Sinkhorn algorithm
  • Remark 4.2
  • ...and 25 more