Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Nonasymptotic Scaling Guarantee of Hyperparameter Estimation in Inhomogeneous, Weakly-Dependent Complex Network Dynamical Systems

Yi Yu, Yubo Hou, Yinchong Wang, Nan Zhang, Jianfeng Feng, Wenlian Lu

TL;DR

The paper tackles the challenge of establishing statistical consistency for hyperparameter estimation in large-scale, inhomogeneous complex network dynamical systems under a hierarchical Bayesian framework. It develops a measure-transport-based formulation with mean-type observations and proves nonasymptotic consistency bounds: first for i.i.d. nodes with a $1/\sqrt{n}$ rate and then for weakly-dependent nodes under $\beta$-mixing with polynomial decay, yielding a rate of $n^{-\lambda/(1+2\lambda)}$ that recovers the IID rate as dependence weakens. The methodology combines concentration-of-measure arguments, independent approximations (Rio), and empirical-process techniques (Dudley’s integral) to derive finite-sample guarantees. Numerical experiments on SIS and Spiking Neural Networks corroborate the theory, showing that estimation error decays as network size grows, thereby supporting the reliability of hierarchical Bayesian inference for large-scale networks in practice.

Abstract

Hierarchical Bayesian models are increasingly used in large, inhomogeneous complex network dynamical systems by modeling parameters as draws from a hyperparameter-governed distribution. However, theoretical guarantees for these estimates as the system size grows have been lacking. A critical concern is that hyperparameter estimation may diverge for larger networks, undermining the model's reliability. Formulating the system's evolution in a measure transport perspective, we propose a theoretical framework for estimating hyperparameters with mean-type observations, which are prevalent in many scientific applications. Our primary contribution is a nonasymptotic bound for the deviation of estimate of hyperparameters in inhomogeneous complex network dynamical systems with respect to network population size, which is established for a general family of optimization algorithms within a fixed observation duration. While we firstly establish a consistency result for systems with independent nodes, our main result extends this guarantee to the more challenging and realistic setting of weakly-dependent nodes. We validate our theoretical findings with numerical experiments on two representative models: a Susceptible-Infected-Susceptible model and a Spiking Neuronal Network model. In both cases, the results confirm that the estimation error decreases as the network population size increases, aligning with our theoretical guarantees. This research proposes the foundational theory to ensure that hierarchical Bayesian methods are statistically consistent for large-scale inhomogeneous systems, filling a gap in this area of theoretical research and justifying their application in practice.

On the Nonasymptotic Scaling Guarantee of Hyperparameter Estimation in Inhomogeneous, Weakly-Dependent Complex Network Dynamical Systems

TL;DR

The paper tackles the challenge of establishing statistical consistency for hyperparameter estimation in large-scale, inhomogeneous complex network dynamical systems under a hierarchical Bayesian framework. It develops a measure-transport-based formulation with mean-type observations and proves nonasymptotic consistency bounds: first for i.i.d. nodes with a rate and then for weakly-dependent nodes under -mixing with polynomial decay, yielding a rate of that recovers the IID rate as dependence weakens. The methodology combines concentration-of-measure arguments, independent approximations (Rio), and empirical-process techniques (Dudley’s integral) to derive finite-sample guarantees. Numerical experiments on SIS and Spiking Neural Networks corroborate the theory, showing that estimation error decays as network size grows, thereby supporting the reliability of hierarchical Bayesian inference for large-scale networks in practice.

Abstract

Hierarchical Bayesian models are increasingly used in large, inhomogeneous complex network dynamical systems by modeling parameters as draws from a hyperparameter-governed distribution. However, theoretical guarantees for these estimates as the system size grows have been lacking. A critical concern is that hyperparameter estimation may diverge for larger networks, undermining the model's reliability. Formulating the system's evolution in a measure transport perspective, we propose a theoretical framework for estimating hyperparameters with mean-type observations, which are prevalent in many scientific applications. Our primary contribution is a nonasymptotic bound for the deviation of estimate of hyperparameters in inhomogeneous complex network dynamical systems with respect to network population size, which is established for a general family of optimization algorithms within a fixed observation duration. While we firstly establish a consistency result for systems with independent nodes, our main result extends this guarantee to the more challenging and realistic setting of weakly-dependent nodes. We validate our theoretical findings with numerical experiments on two representative models: a Susceptible-Infected-Susceptible model and a Spiking Neuronal Network model. In both cases, the results confirm that the estimation error decreases as the network population size increases, aligning with our theoretical guarantees. This research proposes the foundational theory to ensure that hierarchical Bayesian methods are statistically consistent for large-scale inhomogeneous systems, filling a gap in this area of theoretical research and justifying their application in practice.
Paper Structure (11 sections, 14 theorems, 122 equations, 6 figures, 1 table)

This paper contains 11 sections, 14 theorems, 122 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Main Results
  4. Preliminary
  5. Estimation via Observations of Independent System
  6. Estimation via Observations of Weakly-Dependent Complex Network Dynamical System
  7. Numerical Experiments
  8. Data Assimilation in SIS Models
  9. Data Assimilation in Spike Neuronal Networks
  10. Conclusion
  11. Biography Section

Key Result

Lemma 1

If the objective function $L^\star: {\mathcal{H}} \to {\mathbb{R}}$ is identifiable and continuous with respect to ${\mathbf{h}}$, and ${\mathcal{H}}$ is compact, then $L^\star$ is strictly identifiable at the unique minimizer ${\mathbf{h}}^\star$.

Figures (6)

  • Figure 1: The RRMSE of the observation ${\mathbf{Y}}(\tilde{{\mathbf{h}}})$ with perturbed hyperparameter $\tilde{{\mathbf{h}}}$ and the true observation ${\mathbf{Y}}^\star$ with the preset true hyperparameter ${\mathbf{h}}^\star$ for different perturbation noise amplitude, for SIS model experiments.
  • Figure 2: Hyperparameter estimation results for different network population sizes for SIS model experiments. The RAE is computed for the last 100 steps of the estimation process representing the final converged performance.
  • Figure 3: The RAE curves of hyperparameter estimation process for different neural network sizes for SIS model experiments. The top 4 figures (a, b, c, d) show the whole RAE curves of estimation process for 4 groups of experiments with different network population sizes of ([400, 600, 800, 1000, 1200, 1400], [1600, 1800, 2000, 2200, 2400], [2600, 2800, 3000, 3200, 3400], [5000, 6000, 7000, 8000, 9000]), respectively. The bottom 4 figures (e, f, g, h) are the zoom-in illustration of the top 4 figures (a, b, c, d), respectively.
  • Figure 4: The RRMSE of the observation ${\mathbf{Y}}(\tilde{{\mathbf{h}}})$ with perturbed hyperparameter $\tilde{{\mathbf{h}}}$ and the true observation ${\mathbf{Y}}^\star$ with the preset true hyperparameter ${\mathbf{h}}^\star$ for different perturbation noise amplitude, for SNN model experiments.
  • Figure 5: The RAE curves of hyperparameter estimation process for different neural network sizes for SNN experiments. The top 3 figures (a, b, c) show the whole RAE curves of estimation process for 3 groups of experiments with different network population sizes of ([400, 600, 800, 1000, 1200], [1600, 2000, 2400, 2800, 3200], [4000, 5000, 8000, 10000]), respectively. The bottom 3 figures (d, e, f) are the zoom-in illustration of the top 3 figures (a, b, c), respectively.
  • ...and 1 more figures

Theorems & Definitions (36)

  • Remark 1
  • Remark 2
  • Remark 3
  • Definition 1
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Definition 2: Sub-Gaussian Random Variable
  • Proposition 1
  • ...and 26 more