On the Development of Binary Classification Algorithm Based on Principles of Geometry and Statistical Inference

TL;DR

This work proposes a geometry-driven binary classifier, the Moving Points Algorithm (MPA), which learns a separating hyperplane by iteratively moving misclassified points toward the opposite-class mean within an ambient $\mathbb{R}^n$ space. Initialization leverages class means to define a boundary via a line $l_1$ (in 2D) with an explicit update rule based on displacement $d$ and a misclassification metric $\lambda$, then updates boundary direction using vectors that move a migrating point toward the appropriate class. The method scales to higher dimensions by solving for an $n-1$ dimensional hyperplane with coefficients $a_1, \dots, a_n$ through a determinant-based construction and employing the same learning dynamics; performance is benchmarked against KNN, Perceptron, and linear SVM on synthetic and real datasets, showing competitive accuracy and small generalization gaps. The paper also discusses overfitting, convergence issues, and extensions via polynomial interpolation and splines to mitigate Runge’s phenomenon, suggesting a broader toolkit for geometry-based decision boundaries in higher dimensions. Overall, the approach provides a geometry-centered, iterative alternative to standard linear classifiers, with demonstrated feasibility and avenues for refinement and dimensional scalability.

Abstract

The aim of this paper is to investigate an attempt to build a binary classification algorithm using principles of geometry such as vectors, planes, and vector algebra. The basic idea behind the proposed algorithm is that a hyperplane can be used to completely separate a given set of data points mapped to n dimensional space, if the given data points are linearly separable in the n dimensions. Since points are the foundational elements of any geometrical construct, by manipulating the position of points used for the construction of a given hyperplane, the position of the hyperplane itself can be manipulated. The paper includes testing data against other classifiers on a variety of standard machine learning datasets. With a focus on support vector machines, since they and our proposed classifier use the same geometrical construct of hyperplane, and the versatility of SVMs make them a good bench mark for comparison. Since the algorithm focuses on moving the points through the hyperspace to which the dataset has been mapped, it has been dubbed as moving points algorithm.

Figures (7)

• Figure 1: Graphical representation of Calculation of displacement vector
• Figure 2: Two moving points start moving close to each other as the model trains.
• Figure 3: Model trains over a non-linearly separable dataset for 150 epochs
• Figure 4: Same model as above trains over the same dataset for 5000 epochs
• Figure 5: The decision boundary calculated by moving points algorithm on a synthetic dataset.

Theorems & Definitions (4)

• Conjecture 1
• Definition 1
• theorem 1
• proof