Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Euclidean Distance Degree of Quadratic Two-Neuron Neural Networks

Giacomo Graziani

TL;DR

This work analyzes the Euclidean Distance degree ($ED$-degree) for shallow two-neuron quadratic neural networks by embedding the problem in algebraic geometry. It introduces the virtual ED-degree ($vED$) as a projective invariant equal to the sum of polar degrees and uses Kempf resolution, Nash blow-up, and intersection theory to reduce the computation to explicit Grassmannian integrals, proving stable polynomiality in the input dimension via equivariant localization. The authors also demonstrate that the ED-degree depends on the chosen metric, with a numerical example showing a substantial gap between the generic metric and the Bombieri–Weyl metric. The results provide a rigorous framework for understanding metric dependence of ED-based model-selection criteria and offer a scalable method to compute these invariants in higher dimensions.

Abstract

We study the Euclidean Distance degree of algebraic neural network models from the perspective of algebraic geometry. Focusing on shallow networks with two neurons, quadratic activation, and scalar output, we identify the associated neurovariety with the second secant variety of a quadratic Veronese embedding. We introduce and analyze the virtual Euclidean Distance degree, a projective invariant defined as the sum of the polar degrees of the variety, which coincides with the usual Euclidean Distance degree for a generic choice of scalar product. Using intersection theory, Chern-Mather classes, and the Nash blow-up provided by Kempf's resolution, we reduce the computation of the virtual Euclidean Distance degree to explicit intersection numbers on a Grassmannian. Applying equivariant localization, we prove that this invariant depends stably polynomially on the input dimension. Numerical experiments based on homotopy continuation illustrate the dependence of the Euclidean Distance degree on the chosen metric and highlight the distinction between the generic and nongeneric cases, such as the Bombieri-Weyl metric.

On the Euclidean Distance Degree of Quadratic Two-Neuron Neural Networks

TL;DR

This work analyzes the Euclidean Distance degree (-degree) for shallow two-neuron quadratic neural networks by embedding the problem in algebraic geometry. It introduces the virtual ED-degree () as a projective invariant equal to the sum of polar degrees and uses Kempf resolution, Nash blow-up, and intersection theory to reduce the computation to explicit Grassmannian integrals, proving stable polynomiality in the input dimension via equivariant localization. The authors also demonstrate that the ED-degree depends on the chosen metric, with a numerical example showing a substantial gap between the generic metric and the Bombieri–Weyl metric. The results provide a rigorous framework for understanding metric dependence of ED-based model-selection criteria and offer a scalable method to compute these invariants in higher dimensions.

Abstract

We study the Euclidean Distance degree of algebraic neural network models from the perspective of algebraic geometry. Focusing on shallow networks with two neurons, quadratic activation, and scalar output, we identify the associated neurovariety with the second secant variety of a quadratic Veronese embedding. We introduce and analyze the virtual Euclidean Distance degree, a projective invariant defined as the sum of the polar degrees of the variety, which coincides with the usual Euclidean Distance degree for a generic choice of scalar product. Using intersection theory, Chern-Mather classes, and the Nash blow-up provided by Kempf's resolution, we reduce the computation of the virtual Euclidean Distance degree to explicit intersection numbers on a Grassmannian. Applying equivariant localization, we prove that this invariant depends stably polynomially on the input dimension. Numerical experiments based on homotopy continuation illustrate the dependence of the Euclidean Distance degree on the chosen metric and highlight the distinction between the generic and nongeneric cases, such as the Bombieri-Weyl metric.
Paper Structure (17 sections, 7 theorems, 81 equations)

This paper contains 17 sections, 7 theorems, 81 equations.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Conventions and notation
  4. Resolution of $\sigma_2\left( v_2 \left( V \right) \right)$
  5. Neurovarieties with scalar output and the $\mathrm{ED}$-degree
  6. Definitions
  7. Norms and scalar products
  8. The $\mathrm{ED}$-degree and the $\mathrm{ED}$-correspondence
  9. The virtual $\mathrm{ED}$-degree
  10. Stable polynomiality of $v\mathrm{ED}$ in the case of $m=1$ and $r=k=2$
  11. Polar degrees, Chern–Mather classes and Nash blow-up of $\mathcal{M}$
  12. Reduction to integrals on $\mathrm{Gr}_2 \left(\mathbb{C}^n\right)$
  13. Equivariant localization on $\mathrm{Gr}_2 \left(\mathbb{C}^n\right)$ and polynomiality
  14. A question
  15. Numerical results
  16. ...and 2 more sections

Key Result

Lemma 3.6

The Bombieri-Weyl product is a scalar product, that is, it is symmetric, linear in both variables, and positive-definite. Moreover, it does not depend on the choice of the orthonormal basis.

Theorems & Definitions (34)

  • Definition 2.1
  • Remark 2.2
  • Definition 3.1
  • Remark 3.2
  • Definition 3.3
  • Definition 3.4
  • Definition 3.5
  • Lemma 3.6
  • proof
  • Definition 3.7
  • ...and 24 more