Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Alexander-Hirschowitz theorem for neurovarieties

A. Massarenti, M. Mella

TL;DR

The paper studies the algebraic geometry of polynomial neural networks by introducing neurovarieties and deriving conditions under which they achieve their expected dimension. It develops a Terracini-type decomposition using level-wise normal spaces to reduce dimension questions to properties of Veronese secant varieties, yielding an Alexander–Hirschowitz-type theorem for the single-output case ($n_L=1$) and, under numerical assumptions, non-defectiveness and global identifiability for multi-output architectures. The results illuminate how architectural choices (widths $\mathbf n$, activation degrees $\mathbf d$) impact expressivity, identifiability, and conditioning, with concrete criteria and explicit counterexamples. Practical tools, including a Magma library, enable symbolic exploration of dimensions across architectures and support principled network design and compression guided by algebraic constraints.

Abstract

We study neurovarieties for polynomial neural networks and fully characterize when they attain the expected dimension in the single-output case. As consequences, we establish non-defectiveness and global identifiability for multi-output architectures.

The Alexander-Hirschowitz theorem for neurovarieties

TL;DR

The paper studies the algebraic geometry of polynomial neural networks by introducing neurovarieties and deriving conditions under which they achieve their expected dimension. It develops a Terracini-type decomposition using level-wise normal spaces to reduce dimension questions to properties of Veronese secant varieties, yielding an Alexander–Hirschowitz-type theorem for the single-output case () and, under numerical assumptions, non-defectiveness and global identifiability for multi-output architectures. The results illuminate how architectural choices (widths , activation degrees ) impact expressivity, identifiability, and conditioning, with concrete criteria and explicit counterexamples. Practical tools, including a Magma library, enable symbolic exploration of dimensions across architectures and support principled network design and compression guided by algebraic constraints.

Abstract

We study neurovarieties for polynomial neural networks and fully characterize when they attain the expected dimension in the single-output case. As consequences, we establish non-defectiveness and global identifiability for multi-output architectures.

Paper Structure

This paper contains 8 sections, 7 theorems, 101 equations.

Table of Contents

  1. Neurovarieties
  2. Neural networks with $n_L = 1$
  3. Level-wise normal spaces and a Terracini type lemma for neurovarieties
  4. Non defectiveness of neurovarieties
  5. Guiding example
  6. The guiding example revisited
  7. Neurovarieties Magma library
  8. Linear independence for polynomial powers

Key Result

Corollary 2.5

The polynomial neural network is globally identifiable if for a general element $F\in \mathcal{V}_{\mathbf n,\mathbf d}$ there is a unique set of points $\{x_1,\ldots, x_{n_{L-1}}\}\subset \tilde{\mathcal{V}}_{L-1}$ such that $F\in\langle x_1,\ldots,x_{n_{L-1}}\rangle$.

Theorems & Definitions (31)

  • Definition 1.1
  • Remark 1.2
  • Definition 1.3: UDBC25
  • Example 1.5
  • Remark 2.2
  • Definition 2.3: Expected dimension of neurovarieties with $n_L = 1$
  • Remark 2.4
  • Corollary 2.5
  • Remark 2.6
  • Definition 3.1
  • ...and 21 more