Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Networks Asymptotic Behaviours for the Resolution of Inverse Problems

Luigi Del Debbio, Manuel Naviglio, Francesco Tarantelli

TL;DR

The paper investigates inverse (deconvolution) problems in a quantum-field-inspired setting by analyzing neural networks through their asymptotic Gaussian-process limits. It formalizes two main limits—the linear (NTK) limit and the infinite-width (NNGP) limit—and relates them to BG methods, comparing their performance on a lattice harmonic oscillator with a known analytical solution. Across numerical experiments, the Gaussian-process variants (NNGP and NTK-GP) outperform a finite-width fully connected NN, and increasing width drives the NN toward GP behavior, while a non-Bayesian NTK-GP presents a different interpretation of the learned solution. The results suggest that, for this problem, nonlinear finite-width corrections may be unnecessary and motivate further study of training dynamics and architecture extensions (e.g., CNNs) to generalize these findings to more realistic inverse problems.

Abstract

This paper presents a study of the effectiveness of Neural Network (NN) techniques for deconvolution inverse problems relevant for applications in Quantum Field Theory, but also in more general contexts. We consider NN's asymptotic limits, corresponding to Gaussian Processes (GPs), where non-linearities in the parameters of the NN can be neglected. Using these resulting GPs, we address the deconvolution inverse problem in the case of a quantum harmonic oscillator simulated through Monte Carlo techniques on a lattice. In this simple toy model, the results of the inversion can be compared with the known analytical solution. Our findings indicate that solving the inverse problem with a NN yields less performing results than those obtained using the GPs derived from NN's asymptotic limits. Furthermore, we observe the trained NN's accuracy approaching that of GPs with increasing layer width. Notably, one of these GPs defies interpretation as a probabilistic model, offering a novel perspective compared to established methods in the literature. Our results suggest the need for detailed studies of the training dynamics in more realistic set-ups.

Neural Networks Asymptotic Behaviours for the Resolution of Inverse Problems

TL;DR

The paper investigates inverse (deconvolution) problems in a quantum-field-inspired setting by analyzing neural networks through their asymptotic Gaussian-process limits. It formalizes two main limits—the linear (NTK) limit and the infinite-width (NNGP) limit—and relates them to BG methods, comparing their performance on a lattice harmonic oscillator with a known analytical solution. Across numerical experiments, the Gaussian-process variants (NNGP and NTK-GP) outperform a finite-width fully connected NN, and increasing width drives the NN toward GP behavior, while a non-Bayesian NTK-GP presents a different interpretation of the learned solution. The results suggest that, for this problem, nonlinear finite-width corrections may be unnecessary and motivate further study of training dynamics and architecture extensions (e.g., CNNs) to generalize these findings to more realistic inverse problems.

Abstract

This paper presents a study of the effectiveness of Neural Network (NN) techniques for deconvolution inverse problems relevant for applications in Quantum Field Theory, but also in more general contexts. We consider NN's asymptotic limits, corresponding to Gaussian Processes (GPs), where non-linearities in the parameters of the NN can be neglected. Using these resulting GPs, we address the deconvolution inverse problem in the case of a quantum harmonic oscillator simulated through Monte Carlo techniques on a lattice. In this simple toy model, the results of the inversion can be compared with the known analytical solution. Our findings indicate that solving the inverse problem with a NN yields less performing results than those obtained using the GPs derived from NN's asymptotic limits. Furthermore, we observe the trained NN's accuracy approaching that of GPs with increasing layer width. Notably, one of these GPs defies interpretation as a probabilistic model, offering a novel perspective compared to established methods in the literature. Our results suggest the need for detailed studies of the training dynamics in more realistic set-ups.
Paper Structure (14 sections, 56 equations, 3 figures)

This paper contains 14 sections, 56 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Neural Network asymptotic limits
  3. The linear limit
  4. The large width limit
  5. Interplay between linear and infinite width limit
  6. Relation with Backus Gilbert techniques
  7. Numerical experiments
  8. Conclusions
  9. Backus-Gilbert inversion method
  10. Neural Networks basics and Gradient Descent training
  11. Computing the Kernel for NNGP and NKT
  12. Training set details
  13. Harmonic Oscillator
  14. Analytical correlation functions

Figures (3)

  • Figure 1: Comparison of the results obtained using a neural network and its asymptotic limits corresponding to GPs. The dashed green line represents smeared spectral density, smeared with a Gaussian width $\sigma=0.01$. The three methods are trained using the training set described in Appendix \ref{['TrainingApp']} with Gaussian bumps at fixed value of $\sigma=0.01$. The final uncertainties are obtained by bootstrap resampling. In the NTK-GP case they are smaller than the symbols.
  • Figure 2: The figure shows numerically the large width limit of the neural network. Our results clearly show that in the large width limit, $w \rightarrow \infty$, the outcome of the neural network tends to the GPs' ones.
  • Figure 3: The figure shows the spectral functions used as training set, generated as a superposition of Gaussian distributions. The value of $\sigma$ is fixed at 0.01, while $\mu$ is randomly generated within the interval [0,0.3].