Modeling high dimensional point clouds with the spherical cluster model

TL;DR

This work introduces the spherical cluster (SC) model for high-dimensional point clouds, where a center c defines a sphere S(c,r) with radius r set as a fraction η of the variance in distances to c. The optimization problem is strictly convex but non-smooth, and the authors develop an exact solver based on the Clarke gradient operating on an arrangement of spheres, demonstrating substantial speedups over BFGS/L-BFGS in high dimensions. Across diverse datasets (including up to d=10,000), the SC center emerges as a parameterized high-dimensional median, and the theory supports unique solutions and stable behavior as η varies. The results motivate using spherical clusters embedded in affine spaces (SESC) as compact, geometry-aware representations of high-dimensional data, with a companion paper addressing mixtures of SESC components.

Abstract

A parametric cluster model is a statistical model providing geometric insights onto the points defining a cluster. The {\em spherical cluster model} (SC) approximates a finite point set $P\subset \mathbb{R}^d$ by a sphere $S(c,r)$ as follows. Taking $r$ as a fraction $η\in(0,1)$ (hyper-parameter) of the std deviation of distances between the center $c$ and the data points, the cost of the SC model is the sum over all data points lying outside the sphere $S$ of their power distance with respect to $S$. The center $c$ of the SC model is the point minimizing this cost. Note that $η=0$ yields the celebrated center of mass used in KMeans clustering. We make three contributions. First, we show fitting a spherical cluster yields a strictly convex but not smooth combinatorial optimization problem. Second, we present an exact solver using the Clarke gradient on a suitable stratified cell complex defined from an arrangement of hyper-spheres. Finally, we present experiments on a variety of datasets ranging in dimension from $d=9$ to $d=10,000$, with two main observations. First, the exact algorithm is orders of magnitude faster than BFGS based heuristics for datasets of small/intermediate dimension and small values of $η$, and for high dimensional datasets (say $d>100$) whatever the value of $η$. Second, the center of the SC model behave as a parameterized high-dimensional median. The SC model is of direct interest for high dimensional multivariate data analysis, and the application to the design of mixtures of SC will be reported in a companion paper.

Figures (19)

• Figure 1: Minima of $F$ on cells of various dimensions. Data point in orange, minima in red. Selected level sets (in dotted-lines) are also reported.
• Figure 2: LineDescent-NoValue-(-NoValue-)(from $x_0, x_1, x_2$) and SphereDescent-NoValue-(-NoValue-)(from $x_3$) steps. Underlying trajectories are depicted in dark green. Point $y$ is obtained by MinSphereIntersection-NoValue-(-NoValue-)point $x_3$.
• Figure 3: Yeast landsat. This dataset features $6435$ points in dimension $d=9$.
• Figure 4: Spherical cluster: illustrations on a toy 2D dataset.(Left) Trajectories from five different starting points, with $\eta = 0.5$ (Line/Sphere descents in blue/orange). (Right) Evolution of the cluster center for $\eta$ in $[0.1, 0.9]$ be step of 0.1.
• Figure S1: Neighboring cells. Each red dot has $I^0$ of cardinal two and $2^2$ neighboring cells of full dimension. In blue, points with $I^0$ of cardinal 1 have $2^1$ neighboring cells.

Theorems & Definitions (8)

• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof