Linear Convergence of the Subspace Constrained Mean Shift Algorithm: From Euclidean to Directional Data

TL;DR

This work analyzes the algorithmic convergence of the SCMS ridge-finding method by recasting it as a subspace constrained gradient ascent with adaptive steps. It proves linear convergence for both population and sample SCGA/SCMS in Euclidean space and extends the ridge concept to directional data on the unit sphere, establishing a stability theorem for directional ridges on $\\Omega_q$. The directional SCMS algorithm on the sphere achieves analogous linear convergence rates, with geometric considerations (Exp map, geodesic distances) and matrix perturbation results (Davis–Kahan, Weyl) underpinning the theory. Empirical results on synthetic data and earthquakes illustrate the practical benefits of the directional approach, particularly at high latitudes where Euclidean methods suffer bias. The paper advances ridge estimation on manifolds and highlights open issues on bandwidth selection and broader manifold generalizations.

Abstract

This paper studies the linear convergence of the subspace constrained mean shift (SCMS) algorithm, a well-known algorithm for identifying a density ridge defined by a kernel density estimator. By arguing that the SCMS algorithm is a special variant of a subspace constrained gradient ascent (SCGA) algorithm with an adaptive step size, we derive the linear convergence of such SCGA algorithm. While the existing research focuses mainly on density ridges in the Euclidean space, we generalize density ridges and the SCMS algorithm to directional data. In particular, we establish the stability theorem of density ridges with directional data and prove the linear convergence of our proposed directional SCMS algorithm.

Figures (12)

• Figure 1: Density ridges estimated by Euclidean and directional SCMS algorithms on two synthetic datasets (drawn as black points) with hidden circular manifold structures (indicated by blue curves) on $\mathbb{R}^2$ and the unit sphere $\Omega_2 \subset \mathbb{R}^3$, respectively. Left: The orange points indicate the estimated ridge obtained by the Euclidean SCMS algorithm from the dataset on $\mathbb{R}^2$. Right: The red points represent the estimated directional ridge identified by our directional SCMS algorithm, while the orange points indicate the estimated ridge obtained by the Euclidean SCMS algorithm from the dataset on $\Omega_2$. This panel is presented under the Hammer projection; see Appendix \ref{['Sec:Drawback_Euc']} for more details.
• Figure 2: Contour lines of the density function \ref{['den_example']} and its principal gradient flows.
• Figure 3: An illustration of one-step iterations under two candidate directional SCMS algorithms
• Figure 4: Density ridges estimated by the Euclidean SCMS algorithm on the two simulated datasets and their (linear) convergence plots. Horizontally, the first row displays the results of the simulated Gaussian mixture dataset, while the second row presents the results of the half circle simulated dataset. Vertically, the first column includes plots with Euclidean KDE, estimated ridges, and trajectories of SCMS sequences from two (randomly) chosen initial points. The second and third columns present the (linear) convergence plots for the log-distances of points in the highlighted sequences (indicated by hollow cyan points) to their limiting points or the estimated ridges.
• Figure 5: Density ridges estimated by the directional SCMS algorithm performed on the two simulated datasets and their (linear) convergence plots. Horizontally, the first row displays the results on the simulated vMF mixture dataset, while the second row presents the results on the circular simulated dataset on $\Omega_2$. Vertically, the first column includes plots with directional KDE, estimated ridges, and trajectories of directional SCMS sequences from two (randomly) chosen initial points on $\Omega_2$. The second and third columns present the convergence plots for the log-distances of points in the highlighted sequences (indicated by hollow cyan points) to their limiting points or the estimated ridges on $\Omega_2$.

Theorems & Definitions (59)

• Remark 2.1
• Lemma 3.1: Theorem 4 in Non_ridge_est2014
• Remark 3.1
• Lemma 3.2
• Remark 3.2
• Proposition 3.3: Convergence of the SCGA Algorithm
• Corollary 3.4: Convergence of the SCMS Algorithm
• Definition 3.5: Linear Rate of Convergence
• Theorem 3.6: Linear Convergence of the SCGA Algorithm
• Remark 3.3