Combinatorial optimization of the coefficient of determination

Abstract

Robust correlation analysis is among the most critical challenges in statistics. Herein, we develop an efficient algorithm for selecting the $k$- subset of $n$ points in the plane with the highest coefficient of determination $\left( R^2 \right)$. Drawing from combinatorial geometry, we propose a method called the \textit{quadratic sweep} that consists of two steps: (i) projectively lifting the data points into $\mathbb R^5$ and then (ii) iterating over each linearly separable $k$-subset. Its basis is that the optimal set of outliers is separable from its complement in $\mathbb R^2$ by a conic section, which, in $\mathbb R^5$, can be found by a topological sweep in $Θ\left( n^5 \log n \right)$ time. Although key proofs of quadratic separability remain underway, we develop strong mathematical intuitions for our conjectures, then experimentally demonstrate our method's optimality over several million trials up to $n=30$ without error. Implementations in Julia and fully seeded, reproducible experiments are available at https://github.com/marc-harary/QuadraticSweep.

Figures (3)

• Figure 1: Higher-dimensional lifting from $\mathbb R$ to $\mathbb R^2$ linearizes quadratic boundaries, enabling a topological plane sweep. Our boundary is converted from two points $\Gamma_1, \Gamma_2 \in \mathbb R$ to a linear decision boundary $\Gamma < \mathbb R^2$.
• Figure 2: The quadratic decision boundaries $\Gamma$ for each objective function.
• Figure 3: Trial time for the quadratic sweep versus the size of the dataset $n$ for $N=100$ trials

Theorems & Definitions (11)

• Theorem 1
• proof
• Theorem 2
• proof
• Theorem 3
• proof
• Theorem 4
• proof
• Claim 1
• Claim 2