Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Separating balls with partly random hyperplanes with a view to partly random neural networks

Olov Schavemaker

TL;DR

This work derives exact formulas for the probabilities that three partly random hyperplane families $H[w;b]$, $H[w;b]$ and $H[w;b]$ separate two disjoint balls in $\mathbb{R}^n$, linking these results to neural network architectures such as RVFL. The authors obtain closed-form expressions using geometric constructions (a line axis through the balls, double-cone analysis) and special functions (regularized beta functions), showing that fully random hyperplanes underperform compared to partially random variants with either random weights or random biases. A central finding is that random biases or random weights, when optimally aligned, can yield substantially higher separation probabilities than fully random hyperplanes, particularly in high dimensions, suggesting that partially random neural networks may better classify low-dimensional manifolds. The results provide a theoretical step toward studying partially random neural networks and their potential practical advantages in high-dimensional learning tasks.

Abstract

We derive exact expressions for the probabilities that partly random hyperplanes separate two Euclidean balls. The probability that a fully random hyperplane separates two balls turns out to be significantly smaller than the corresponding probabilities for hyperplanes which are not fully random in certain cases. Our results motivate studying partially random neural networks and provide a first step in this direction.

Separating balls with partly random hyperplanes with a view to partly random neural networks

TL;DR

This work derives exact formulas for the probabilities that three partly random hyperplane families , and separate two disjoint balls in , linking these results to neural network architectures such as RVFL. The authors obtain closed-form expressions using geometric constructions (a line axis through the balls, double-cone analysis) and special functions (regularized beta functions), showing that fully random hyperplanes underperform compared to partially random variants with either random weights or random biases. A central finding is that random biases or random weights, when optimally aligned, can yield substantially higher separation probabilities than fully random hyperplanes, particularly in high dimensions, suggesting that partially random neural networks may better classify low-dimensional manifolds. The results provide a theoretical step toward studying partially random neural networks and their potential practical advantages in high-dimensional learning tasks.

Abstract

We derive exact expressions for the probabilities that partly random hyperplanes separate two Euclidean balls. The probability that a fully random hyperplane separates two balls turns out to be significantly smaller than the corresponding probabilities for hyperplanes which are not fully random in certain cases. Our results motivate studying partially random neural networks and provide a first step in this direction.
Paper Structure (6 sections, 4 theorems, 34 equations, 1 figure)

This paper contains 6 sections, 4 theorems, 34 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Setting
  3. Random biases
  4. Random weights
  5. Fully random case
  6. Discussion

Key Result

Theorem \ref{sec:random_biases}

Let $\mathscr{b}$ be as in §sec:setting. Then where $\mathbb{S}^{n-1}\ni\omega$ is parallel to the line through $c$ and $x,$ which we will call $a.$

Figures (1)

  • Figure 1: Binary classification decision boundaries for RVFL (top) and RVFL with learned biases (bottom) with ReLU (left) and hardtanh (right) activation functions. Figure courtesy of Guido Wagenvoorde, Mathematics Department, Utrecht University (master student).

Theorems & Definitions (10)

  • Definition
  • Theorem \ref{sec:random_biases}
  • proof
  • Theorem \ref{sec:random_weights}
  • Remark
  • proof
  • Theorem \ref{sec:fully_random}
  • proof
  • Lemma
  • proof