A De-singularity Subgradient Approach for the Extended Weber Location Problem

TL;DR

The paper addresses singularity in the extended Weber location problem for $1\le q<2$ by introducing a de-singularity subgradient method (qPWAWS) that replaces the ill-defined gradient at data points with a descent direction. It provides a complete convergence proof, establishes superlinear convergence when the minimum lies at a data point, and derives practical update rules for both non-coincidence and coincidence cases. Empirical results on online portfolio selection datasets show that qPWAWS exits singularities efficiently, maintains monotone convergence, and, for certain $q$ values in $(1,2)$, can outperform traditional $q=1$ or $q=2$ medians in investing performance. Overall, the work advances both theory and practice for robust, fast optimization under singularity in the extended Weber problem and offers avenues for broader applicability to related problems.

Abstract

The extended Weber location problem is a classical optimization problem that has inspired some new works in several machine learning scenarios recently. However, most existing algorithms may get stuck due to the singularity at the data points when the power of the cost function $1\leqslant q<2$, such as the widely-used iterative Weiszfeld approach. In this paper, we establish a de-singularity subgradient approach for this problem. We also provide a complete proof of convergence which has fixed some incomplete statements of the proofs for some previous Weiszfeld algorithms. Moreover, we deduce a new theoretical result of superlinear convergence for the iteration sequence in a special case where the minimum point is a singular point. We conduct extensive experiments in a real-world machine learning scenario to show that the proposed approach solves the singularity problem, produces the same results as in the non-singularity cases, and shows a reasonable rate of linear convergence. The results also indicate that the $q$-th power case ($1<q<2$) is more advantageous than the $1$-st power case and the $2$-nd power case in some situations. Hence the de-singularity subgradient approach is beneficial to advancing both theory and practice for the extended Weber location problem.

Figures (7)

• Figure 1: The singularity problem for the extended Weber location problem (\ref{['eqn:lqmedian']}) with $1\leqslant q<2$: (a) The gradient $\nabla C_q(\mathbf{y})$ is well-defined when $\mathbf{y}\notin\{\mathbf{x}_i\}_{i=1}^m$. (b) The gradient $\nabla C_q(\mathbf{y})$ does not necessarily exist when $\mathbf{y}$ hits some $\mathbf{x}_k$.
• Figure 2: $-\nabla D_q(\mathbf{x}_k)$ can be interpreted as the resultant implemented on $\mathbf{x}_k$ towards other data points.
• Figure 3: Theorem \ref{['thm:singmin']} indicates that a small displacement from $\mathbf{x}_k$ towards $-\nabla D_q(\mathbf{x}_k)$ can reduce the $q$-th power cost. Thus we can start from a $\lambda_0$ and reduce it with a factor $\rho<1$ at each time, until $\mathbf{x}_k-\lambda_* \nabla D_q(\mathbf{x}_k)$ reduces the cost.
• Figure 4: Sequences of rates of convergence $\left\{\frac{\|\mathbf{y}_{(p+1)}-\mathbf{y}_{(o)}\|}{\|\mathbf{y}_{(p)}-\mathbf{y}_{(o)}\|}\right\}_{p=1}^{o-2}$ for $q$PWAWS with $q=1.1\sim1.9$ and $m=5$.
• Figure 5: $q$-th power Weiszfeld algorithm without singularity ($q$PWAWS)

Theorems & Definitions (25)

• Definition 1: rockafellar2009variational
• Definition 2: Subdifferential in Convex Analysis
• Theorem 3
• Corollary 4
• Definition 5: $\mathbf{q}$-th Power De-singularity Subgradient
• Theorem 6: Characterization of Subgradients and Minimum
• Theorem 7: De-singularity Subgradient Descent Method
• Lemma 8
• Lemma 9
• Lemma 10