Confidence Sets for Multidimensional Scaling

TL;DR

It is found that the multiplier bootstrap adapts automatically to heteroscedastic noise such as multiplicative noise, while the empirical bootstrap seems to require homoscedasticity.

Abstract

We develop a formal statistical framework for classical multidimensional scaling (CMDS) applied to noisy dissimilarity data. We establish distributional convergence results for the embeddings produced by CMDS for various noise models, which enable the construction of \emph{bona~fide} uniform confidence sets for the latent configuration, up to rigid transformations. We further propose bootstrap procedures for constructing these confidence sets and provide theoretical guarantees for their validity. We find that the multiplier bootstrap adapts automatically to heteroscedastic noise such as multiplicative noise, while the empirical bootstrap seems to require homoscedasticity. Either form of bootstrap, when valid, is shown to substantially improve finite-sample accuracy. The empirical performance of the proposed methods is demonstrated through numerical experiments.

Figures (5)

• Figure 1: Classical multidimensional scaling (MDS) embedding of the noisy pairwise distance between 30 U.S. cities with multiplicative noise. The red points denote the true city locations $X$, while the black points represent the MDS estimates $\widehat{X}$ after Procrustes alignment. The gray ellipsoids constitute the $90\%$ confidence set, which guarantees that each true location lies within its corresponding ellipsoid with probability $1-\alpha=0.9$.
• Figure 2: For $n \in \qty{250, 500, 2000}$ and $p = 5$, latent configurations $X \in \mathds{R}^{n \times p}$ from the same distribution are generated, and the noisy dissimilarities $D = \Delta(X) + \mathcal{E}$ are generated under the additive noise model for ${\varepsilon_{ij}\!\sim_\textup{iid}{}\!N(0, 5)}$. (Left) The empirical c.d.f. of $(\widehat{T}_n - b_n)/a_n$ is shown alongside the c.d.f. of the Gumbel distribution (Center) The kernel density estimates for the same data are compared against the p.d.f. of the Gumbel distribution. (Right) The QQ plot of the empirical quantiles vs. the Gumbel quantiles. Based on $2000$ Monte Carlo trials.
• Figure 3: For the same data in \ref{['fig:gumbel']}, we perform the multiplier bootstrap procedure using $B=4000$ replicates. (Left) The empirical c.d.f. of $(T_n^{\flat} - b_n)/a_n$ is compared to the c.d.f. of $(\widehat{T}_n-b_n)/a_n$. (Center) The kernel density estimate based on the same bootstrap replicates is illustrated alongside the estimates from \ref{['fig:gumbel']}. The Gumbel c.d.f. and p.d.f. are shown in both figures for reference. (Right) The QQ plot of the empirical quantiles of $(T_n^{\flat} - b_n)/a_n$ vs. the empirical quantiles of $(\widehat{T}_n - b_n)/a_n$. Based on $2000$ Monte Carlo trials.
• Figure 4: Coverage probabilities for the multiplier bootstrap and the empirical bootstrap for different noise models. For $N = 20$ different configurations, $X \in \mathds{R}^{n \times 2}$, noisy dissimilarities $D$ are obtained using (left) multiplicative noise and (right) additive noise. Bootstrap confidence sets are computed using both the multiplier bootstrap and the empirical bootstrap procedures for a range of nominal levels $\alpha \in (0, 1)$, and the coverage probabilities are computed across $500$ Monte Carlo runs. Each of the $N$ thin lines correspond to the coverage probabilities obtained for a particular fixed configuration $X$, and the thick lines correspond to the average coverage across all configurations.
• Figure 5: Adaptivity of the multiplier bootstrap confidence sets to heteroscedasticity. Points are sampled from a configuration $X \in \mathds{R}^{n \times p}$ and various noise models, the embedding $\widehat{X} \in \mathds{R}^{n \times p}$ is obtained via the classical MDS algorithm (black $\bullet$) and the confidence sets (grey ellipsoids) are computed using \ref{['alg:multiplier-bootstrap']}. (a) The noise is additive for in the figure on top and multiplicative in the figure below. The latent configuration is shown in red ($\blacktriangledown$). (b) For each pair of points, the noise variance depends on: (top) the sum of each point's squared norm and (bottom) the absolute difference of each point's squared norm. (c) The noise variance depends on: (top) the vertical pairwise distances and (bottom) the horizontal pairwise distances.

Theorems & Definitions (32)

• Remark 2.1
• Remark 2.2
• Theorem 3.1
• proof : Proof Sketch
• Corollary 3.1
• Corollary 3.2
• Proposition 3.1
• Theorem 4.1
• Corollary 4.1
• Proposition 4.1