Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Blessing of dimensionality in cross-validated bandwidth selection on the sphere

José E. Chacón, Eduardo García-Portugués, Andrea Meilán-Vila

TL;DR

The authors address the challenge of data-driven smoothing in directional KDE on the sphere by establishing the existence of a non-degenerate MISE-optimal bandwidth and deriving the exact relative convergence rate of least-squares CV to this benchmark, $n^{-d/(2d+8)}$. They provide a full asymptotic variance analysis for the CV-based selector, yielding a clear expression for the limiting distribution and the dependence on dimension and concentration, explicitly illustrating the blessing of dimensionality as $d$ grows. The work extends Euclidean smoothing theory to non-Euclidean spaces, shows consistency and finite-sample validity through ACV, and demonstrates, via extensive numerical experiments with vMF and mixture models, that CV becomes increasingly competitive and often superior to plug-in methods in higher dimensions. Practically, this supports using CV for bandwidth selection in high-dimensional directional data and clarifies the asymptotic benefits of dimensionality in smoothing parameter choice.

Abstract

We study the asymptotic behavior of least-squares cross-validation bandwidth selection in kernel density estimation on the $d$-dimensional hypersphere, $d\geq 1$. We show that the exact rate of convergence with respect to the optimal bandwidth minimizing the mean integrated squared error, shown to exist under mild non-uniformity conditions, is $n^{-d/(2d+8)}$, thus approaching the $n^{-1/2}$ parametric rate as $d$ grows. This ``blessing of dimensionality'' in bandwidth selection offers theoretical support for utilizing the conceptually simpler cross-validation selector over plug-in techniques for larger dimensions $d$. We compare this result for bandwidth estimation on the $d$-dimensional Euclidean space through explicit expressions for the asymptotic variance functionals. Numerical experiments corroborate the speed of this convergence in an array of scenarios and dimensions, precisely illustrating the tipping dimension where cross-validation outperforms plug-in approaches.

Blessing of dimensionality in cross-validated bandwidth selection on the sphere

TL;DR

The authors address the challenge of data-driven smoothing in directional KDE on the sphere by establishing the existence of a non-degenerate MISE-optimal bandwidth and deriving the exact relative convergence rate of least-squares CV to this benchmark, . They provide a full asymptotic variance analysis for the CV-based selector, yielding a clear expression for the limiting distribution and the dependence on dimension and concentration, explicitly illustrating the blessing of dimensionality as grows. The work extends Euclidean smoothing theory to non-Euclidean spaces, shows consistency and finite-sample validity through ACV, and demonstrates, via extensive numerical experiments with vMF and mixture models, that CV becomes increasingly competitive and often superior to plug-in methods in higher dimensions. Practically, this supports using CV for bandwidth selection in high-dimensional directional data and clarifies the asymptotic benefits of dimensionality in smoothing parameter choice.

Abstract

We study the asymptotic behavior of least-squares cross-validation bandwidth selection in kernel density estimation on the -dimensional hypersphere, . We show that the exact rate of convergence with respect to the optimal bandwidth minimizing the mean integrated squared error, shown to exist under mild non-uniformity conditions, is , thus approaching the parametric rate as grows. This ``blessing of dimensionality'' in bandwidth selection offers theoretical support for utilizing the conceptually simpler cross-validation selector over plug-in techniques for larger dimensions . We compare this result for bandwidth estimation on the -dimensional Euclidean space through explicit expressions for the asymptotic variance functionals. Numerical experiments corroborate the speed of this convergence in an array of scenarios and dimensions, precisely illustrating the tipping dimension where cross-validation outperforms plug-in approaches.
Paper Structure (20 sections, 24 theorems, 256 equations, 10 figures, 2 tables)

This paper contains 20 sections, 24 theorems, 256 equations, 10 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Kernel density estimation
  4. Mean integrated squared error
  5. Existence of the optimal bandwidth
  6. Cross-validation bandwidth selector
  7. Consistency
  8. Convergence rate
  9. Comparison with the Euclidean case
  10. Numerical experiments
  11. Von Mises--Fisher distribution
  12. Mixtures of von Mises--Fisher distributions
  13. Comparison with plug-in bandwidths
  14. Density error rates
  15. Discussion
  16. ...and 5 more sections

Key Result

Theorem 2.1

Assume that $f$ is square integrable and $L$ is continuous, $L(0)>0$ and satisfies A2. Then, there exists $\nu_{\rm MISE}\geq0$ such that ${\rm MISE}2(\nu_{\rm MISE})\leq{\rm MISE}2(\nu)$, for all $\nu\geq0$.

Figures (10)

  • Figure 1: Asymptotic variance functionals as a function of the dimension $d$. Figure \ref{['fig:sigma2:tau']} shows the contribution of the vMF kernel to the asymptotic variance, while Figure \ref{['fig:sigma2:rho']} gives the vMF density contribution for several concentrations $\kappa$. Figure \ref{['fig:sigma2:sigma2']} collects the resulting asymptotic variance factor $\sigma^2_d(L_\mathrm{vMF},\kappa)$. Dotted lines represent the asymptotic approximations as $d\to\infty$. The gray curve in Figure \ref{['fig:sigma2:rho']} shows $d\mapsto\rho_d(\phi)$ from Section \ref{['sec:comp']}. Global extrema of the curves are highlighted with dots. Both axes are $\log_{10}$-scaled.
  • Figure 2: Curves $n\mapsto\hat{\mathrm{e}}(n,d)$ (Figure \ref{['fig:avg-rmse:avg']}) and $n\mapsto\widehat{\mathrm{rmse}}(n,d)$ (Figure \ref{['fig:avg-rmse:rmse']}) for dimensions $d=1,2,\ldots,10$. A $\log_2$-scale is used in the horizontal axes of both panels, and also for the vertical axis of the right panel. Linear model fits for $\{(\log_2(\widehat{\mathrm{rmse}}(n,d)),\log_2(n))\}_{n\geq n_0}$ with $n_0=128$ are shown with dashed lines (see Table \ref{['tab:linfit:vMF']} for their summaries).
  • Figure 3: Evaluation of the asymptotic normality of $n^{-\beta_*(d)}R(n,d)$ for sample sizes $n=2^\ell$, $\ell=5,5.5,\ldots,13$ and dimensions $d=1,2,\ldots,10$. The curves show the kernel density estimates for the sample $\{n^{-\beta_*(d)}R^{(j)}(n,d)\}_{j=1}^M$ featuring normal scale bandwidths.
  • Figure 4: In Figure \ref{['fig:rmse-mise:mise']}, curves $h\mapsto \mathrm{MISE}\{\hat{f}(\cdot;h)\}$ for dimensions $d=1,10$ and sample sizes $n=32,64,\ldots,1024$ showing the varying degree of identifiability of the global minima in the MvMF distribution. In Figure \ref{['fig:rmse-mise:rmse']}, curves $n\mapsto\widehat{\mathrm{rmse}}(n,d)$ for dimensions $d=1,2,\ldots,10$. A $\log_2$-scale is used in both axes. Linear model fits for $\{(\log_2(\widehat{\mathrm{rmse}}(n,d)),\log_2(n))\}_{n\geq n_0}$ with $n_0=512$ are shown with dashed lines (see Table \ref{['tab:linfit:MvMF']} for their summaries).
  • Figure 5: Error curves $n\mapsto \mathbb{E}[\|\hat{f}(\cdot;\hat{h})-g(\cdot;\boldsymbol\theta_0)\|_2]$ for the bandwidth selectors $\hat{h}\in\{\hat{h}_\mathrm{CV},\hat{h}_\mathrm{AMI},\hat{h}_\mathrm{EMI},\hat{h}_\mathrm{ISE}\}$. These curves are benchmarked with respect to the parametric error curve $n\mapsto \mathbb{E}[\|g(\cdot;\hat{\boldsymbol\theta})-g(\cdot;\boldsymbol\theta_0)\|_2]$. The density $g(\cdot;\boldsymbol\theta_0)$ is the vMF from Section \ref{['sec:num:vMF']} and $\hat{\boldsymbol\theta}$ is the maximum likelihood estimator.
  • ...and 5 more figures

Theorems & Definitions (48)

  • Theorem 2.1
  • Theorem 2.2
  • Lemma 3.1
  • Lemma 3.2
  • Lemma 3.3
  • Theorem 3.1
  • Lemma 3.4
  • Theorem 3.2
  • Corollary 3.1
  • Corollary 3.2
  • ...and 38 more