Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Kolmogorov--Arnold stability

Sviatoslav V. Dzhenzher, Michael H. Freedman

TL;DR

The paper analyzes the robustness of Kolmogorov--Arnold (KA) representations when hidden-space re-parameterizations are allowed, framing re-parameterizations as adversarial actions on the network's inner layer. Using Baire-category arguments, it proves KA stability for countable collections of continuous re-parameterizations by constructing a tuple $\phi$ and a uniformly continuous outer function $g$ such that any target function $f:I^n\to\mathbb{R}$ satisfies $f=\operatorname{Approx}(g,h)$ for each $h$ in the admissible set; a key step is showing openness and density of suitable approximation sets $U_{f,h}$ and performing a contraction-based limit. It further extends the result to multiple inner maps, showing that $f(x_1,...,x_n)=\sum_{j=1}^{2n+1} g\big(h^1_j(\phi^1(x_1))+...+h^n_j(\phi^n(x_n))\big)$ can be achieved for any $f$ and $h^i$ in the countable family. However, the authors identify a fundamental obstruction: equi-continuity of the outer functions $g^{f,h}$ fails to hold uniformly over continuous families of re-parameterizations, preventing a straightforward limit to the full homeomorphism group and leaving open questions about the ultimate applicability of KA to general neural-network theory and computability of all functions.

Abstract

Regarding the representation theorem of Kolmogorov and Arnold (KA) as an algorithm for representing or <<expressing>> functions, we test its robustness by analyzing its stability to withstand re-parameterizations of the hidden space. One may think of such re-parameterizations as the work of an adversary attempting to foil the construction of the KA outer function. We find KA to be stable under countable collections of continuous re-parameterizations, but unearth a question about the equi-continuity of the outer functions that, so far, obstructs taking limits and defeating continuous groups of re-parameterizations. This question on the regularity of the outer functions is relevant to the debate over the applicability of KA to the general theory of NNs.

Kolmogorov--Arnold stability

TL;DR

The paper analyzes the robustness of Kolmogorov--Arnold (KA) representations when hidden-space re-parameterizations are allowed, framing re-parameterizations as adversarial actions on the network's inner layer. Using Baire-category arguments, it proves KA stability for countable collections of continuous re-parameterizations by constructing a tuple and a uniformly continuous outer function such that any target function satisfies for each in the admissible set; a key step is showing openness and density of suitable approximation sets and performing a contraction-based limit. It further extends the result to multiple inner maps, showing that can be achieved for any and in the countable family. However, the authors identify a fundamental obstruction: equi-continuity of the outer functions fails to hold uniformly over continuous families of re-parameterizations, preventing a straightforward limit to the full homeomorphism group and leaving open questions about the ultimate applicability of KA to general neural-network theory and computability of all functions.

Abstract

Regarding the representation theorem of Kolmogorov and Arnold (KA) as an algorithm for representing or <<expressing>> functions, we test its robustness by analyzing its stability to withstand re-parameterizations of the hidden space. One may think of such re-parameterizations as the work of an adversary attempting to foil the construction of the KA outer function. We find KA to be stable under countable collections of continuous re-parameterizations, but unearth a question about the equi-continuity of the outer functions that, so far, obstructs taking limits and defeating continuous groups of re-parameterizations. This question on the regularity of the outer functions is relevant to the debate over the applicability of KA to the general theory of NNs.

Paper Structure

This paper contains 6 sections, 6 theorems, 40 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Background and statements
  3. Proof of Theorem \ref{['t:ka']}
  4. Proof of Theorem \ref{['t:many-adv']}
  5. Related work and positioning
  6. Conclusions

Key Result

Theorem 2.1

Let $n>1$ be an integer. Let $\mathcal{H}$ be a countable set of homeomorphisms $\mathbb{R}^{2n+1}\to\mathbb{R}^{2n+1}$. For any rationally independent $\gamma_1,\ldots,\gamma_n\in\mathbb{R}$ there exists a tuple $\phi\colon I\to I^{2n+1}$ such that for any $f\colon I^n\to\mathbb{R}$ and any homeomo

Figures (3)

  • Figure 1: KA-style network with a homeomorphism $h$ acting on each input neuron in parallel
  • Figure 2: KA-style network with a homeomorphism $h$ acting on the sum of all neuron inputs simultaneously
  • Figure 3: Commuting of the 'extension' from one to two variables and the affine transform $A$

Theorems & Definitions (12)

  • Theorem 2.1: proved in §\ref{['s:proof-1']} below
  • Theorem 2.2: proved in §\ref{['s:proof-2']} below
  • Lemma 3.1
  • proof
  • Lemma 3.2
  • proof
  • proof : Proof of Theorem \ref{['t:ka']}
  • Lemma 4.1
  • proof
  • Lemma 4.2
  • ...and 2 more