Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dimensionality reduction and width of deep neural networks based on topological degree theory

Xiao-Song Yang

TL;DR

The paper addresses how the topological structure of high-dimensional data constrains dimensionality reduction by deep neural networks. It builds a framework using link theory and group-homotopy to show that, for disjoint embedded datasets forming a linked pair with nonzero degree, any width-$\le n$ network cannot separate them, and that universal approximation on the unit ball requires width exceeding the input dimension $n$. The results hold for activation functions that are group-homotopic to a homeomorphism (including Relu and sigmoid-based activations) and yield concrete width lower bounds, offering a geometric explanation for the necessity of sufficient width in autoencoders and dimensionality-reduction tasks. Collectively, the work connects topological invariants to neural-network architecture, with potential implications for data geometry-aware network design and analysis.

Abstract

In this paper we present a mathematical framework on linking of embeddings of compact topological spaces into Euclidean spaces and separability of linked embeddings under a specific class of dimension reduction maps. As applications of the established theory, we provide some fascinating insights into classification and approximation problems in deep learning theory in the setting of deep neural networks.

Dimensionality reduction and width of deep neural networks based on topological degree theory

TL;DR

The paper addresses how the topological structure of high-dimensional data constrains dimensionality reduction by deep neural networks. It builds a framework using link theory and group-homotopy to show that, for disjoint embedded datasets forming a linked pair with nonzero degree, any width- network cannot separate them, and that universal approximation on the unit ball requires width exceeding the input dimension . The results hold for activation functions that are group-homotopic to a homeomorphism (including Relu and sigmoid-based activations) and yield concrete width lower bounds, offering a geometric explanation for the necessity of sufficient width in autoencoders and dimensionality-reduction tasks. Collectively, the work connects topological invariants to neural-network architecture, with potential implications for data geometry-aware network design and analysis.

Abstract

In this paper we present a mathematical framework on linking of embeddings of compact topological spaces into Euclidean spaces and separability of linked embeddings under a specific class of dimension reduction maps. As applications of the established theory, we provide some fascinating insights into classification and approximation problems in deep learning theory in the setting of deep neural networks.

Paper Structure

This paper contains 7 sections, 15 theorems, 64 equations.

Table of Contents

  1. Introduction
  2. Links and link pairs
  3. Link in terms of homotopy
  4. A special homotopy
  5. Inseparability of linked compact sets under a class of dimension reduction maps
  6. Applications to problems in deep learning
  7. A summary

Key Result

Proposition 2.4

Suppose that there are continuous maps $\phi:S^{k}\to A$ and $\varphi:S^{h}\to B$ with $k+h=n-1$ such that $f\circ\phi:S^{k}\rightarrow\mathbb{R}^{n}$ and $g\circ\varphi:S^{h}\rightarrow\mathbb{R}^{n}$ are linked, then $f(A)$ and $g(B)$ are a link pair.

Theorems & Definitions (33)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Proposition 2.4
  • proof
  • Proposition 2.5
  • Definition 3.1
  • Proposition 3.2
  • proof
  • Proposition 3.3
  • ...and 23 more