Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Local Identifiability of Fully-Connected Feed-Forward Networks with Nonlinear Node Dynamics

Martina Vanelli, Julien M. Hendrickx

TL;DR

This work addresses the identifiability of weights in nonlinear network systems with linear interconnections and nonlinear node dynamics, under partial excitation and partial measurement. By fixing the graph topology and node nonlinearity, it proves that fully-connected layered feed-forward networks are generically locally identifiable for the class of analytic activations with $f(0)=0$ when sources are excited and sinks are measured, leveraging a two-tier genericity in the weight matrix and the function class. The proof combines a finite-dimensional reduction via Maclaurin coefficients with a constructive exponential example to establish identifiability, and shows that this identifying power persists for almost all admissible $W$ and $f$. This result broadens prior identifiability conditions by avoiding the need to excite/measure every node and has implications for neural networks and nonlinear network identification in practical sensing scenarios.

Abstract

We study the identifiability of nonlinear network systems with partial excitation and partial measurement when the network dynamics is linear on the edges and nonlinear on the nodes. We assume that the graph topology and the nonlinear functions at the node level are known, and we aim to identify the weight matrix of the graph. Our main result is to prove that fully-connected layered feed-forward networks are generically locally identifiable by exciting sources and measuring sinks in the class of analytic functions that cross the origin. This holds even when all other nodes remain unexcited and unmeasured and stands in sharp contrast to most findings on network identifiability requiring measurement and/or excitation of each node. The result applies in particular to feed-forward artificial neural networks with no offsets and generalizes previous literature by considering a broader class of functions and topologies.

Local Identifiability of Fully-Connected Feed-Forward Networks with Nonlinear Node Dynamics

TL;DR

This work addresses the identifiability of weights in nonlinear network systems with linear interconnections and nonlinear node dynamics, under partial excitation and partial measurement. By fixing the graph topology and node nonlinearity, it proves that fully-connected layered feed-forward networks are generically locally identifiable for the class of analytic activations with when sources are excited and sinks are measured, leveraging a two-tier genericity in the weight matrix and the function class. The proof combines a finite-dimensional reduction via Maclaurin coefficients with a constructive exponential example to establish identifiability, and shows that this identifying power persists for almost all admissible and . This result broadens prior identifiability conditions by avoiding the need to excite/measure every node and has implications for neural networks and nonlinear network identification in practical sensing scenarios.

Abstract

We study the identifiability of nonlinear network systems with partial excitation and partial measurement when the network dynamics is linear on the edges and nonlinear on the nodes. We assume that the graph topology and the nonlinear functions at the node level are known, and we aim to identify the weight matrix of the graph. Our main result is to prove that fully-connected layered feed-forward networks are generically locally identifiable by exciting sources and measuring sinks in the class of analytic functions that cross the origin. This holds even when all other nodes remain unexcited and unmeasured and stands in sharp contrast to most findings on network identifiability requiring measurement and/or excitation of each node. The result applies in particular to feed-forward artificial neural networks with no offsets and generalizes previous literature by considering a broader class of functions and topologies.

Paper Structure

This paper contains 12 sections, 9 theorems, 36 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Problem formulation
  3. Some fundamental examples and local identifiability
  4. Main result
  5. Proof of the main result
  6. Genericity in the weight matrix
  7. Genericity in the functions
  8. Exponential example
  9. Conclusions
  10. Proof of Lemma \ref{['lemma:gen_W']}
  11. Proof of Lemma \ref{['lemma:F_analytic']}
  12. Proof of Lemma \ref{['lemma2']}

Key Result

Theorem 1

Let $\mathcal{G}$ be a fully-connected LFN and let $\mathcal{F}_Z$ be as in eq:FZ. Then, the network $\mathcal{G}$ is generically locally identifiable in the class $\mathcal{F}_Z$ by exciting the sources and measuring the sinks.

Figures (2)

  • Figure 1: Nonlinear node dynamics where nodes $1$ and $2$ are excited and node $4$ is measured.
  • Figure 2: Graph considered in Example \ref{['ex:ann']}

Theorems & Definitions (31)

  • Example 1
  • Definition 1: Identifiability of a network
  • Example 2
  • Example 3
  • Definition 2: Local identifiability
  • Definition 3
  • Definition 4
  • Example 4
  • Definition 5
  • Theorem 1
  • ...and 21 more