Distribution-on-Distribution Regression with Wasserstein Metric: Multivariate Gaussian Case

TL;DR

This study introduces models for regression from one Gaussian distribution to another, utilizing the Wasserstein metric, and establishes the convergence rates of in-sample prediction errors for the empirical risk minimizations in the models.

Abstract

Distribution data refers to a data set where each sample is represented as a probability distribution, a subject area receiving burgeoning interest in the field of statistics. Although several studies have developed distribution-to-distribution regression models for univariate variables, the multivariate scenario remains under-explored due to technical complexities. In this study, we introduce models for regression from one Gaussian distribution to another, utilizing the Wasserstein metric. These models are constructed using the geometry of the Wasserstein space, which enables the transformation of Gaussian distributions into components of a linear matrix space. Owing to their linear regression frameworks, our models are intuitively understandable, and their implementation is simplified because of the optimal transport problem's analytical solution between Gaussian distributions. We also explore a generalization of our models to encompass non-Gaussian scenarios. We establish the convergence rates of in-sample prediction errors for the empirical risk minimizations in our models. In comparative simulation experiments, our models demonstrate superior performance over a simpler alternative method that transforms Gaussian distributions into matrices. We present an application of our methodology using weather data for illustration purposes.

Figures (7)

• Figure 1: Illustration of structure of the proposed regression model between the Gaussain spaces $\mathcal{G}(\mathbb{R}^{d_1})$ and $\mathcal{G}(\mathbb{R}^{d_2})$. The Gaussian distributions $\nu_1$ and $\nu_2$ are transformed to the random elements $X$ and $Y$ in the linear matrix spaces $\Xi_{d_1}$ and $\Xi_{d_2}$ by the nearly isometric maps $\varphi_{\nu_{1\oplus}}$ and $\varphi_{\nu_{2\oplus}}$, respectively. Then, linear regression model with regression map $\Gamma_{\mathbb{B}_0}$ is assumed between $X$ and $Y$.
• Figure 2: Boxplots of the out-of-sample AWDs defined as \ref{['eq:awd']} for the four scenarios with $n \in \{50, 200\}$ and $N \in \{50, 500\}$. "proposed" denotes the proposed method and "alternative" denotes the alternative method. The number in brackets "[ ]" below the boxplots for the proposed indicates how many runs event \ref{['eq:awd']} happened and boundary projection was needed.
• Figure 3: Boxplots of the out-of-sample AWDs defined as \ref{['eq:awd']} for the low-rank methods with rank $K \in \{2, 3, 4\}$. "proposed" denotes the proposed method and "alternative" denotes the alternative method. The number in brackets "[ ]" below the boxplots for the proposed indicates how many runs event \ref{['eq:awd']} happened and boundary projection was needed.
• Figure 4: Boxplots of the out-of-sample AWDs defined as \ref{['eq:awd']} for the three scenarios with the degree of the freedom $\ell \in \{5, 10, 15\}$. "proposed" denotes the proposed method and "alternative" denotes the alternative method. The number in brackets "[ ]" below the boxplots for the proposed indicates how many runs event \ref{['eq:fall']} happened and boundary projection was needed.
• Figure 5: Observed data and estimated Gaussian joint densities of the average temperatures and average relative humidity in spring (top row) and summer (bottom row) from 1953 to 1956. Black points are observed data and solid lines are contour lines of estimated densities.

Theorems & Definitions (13)

• Proposition 1
• Remark 1: Scalar response model
• Theorem 1: Basic Model
• Theorem 2: Rank-$K$ Model
• proof : Proof of Proposition 1
• Theorem 3: park2023towards, Section 4.1
• proof : Proof of Theorem 1
• proof : Proof of Theorem 2
• Theorem 4: Consistency of Estimator
• proof