Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Partial Differential Equations is All You Need for Generating Neural Architectures -- A Theory for Physical Artificial Intelligence Systems

Ping Guo, Kaizhu Huang, Zenglin Xu

TL;DR

It is of significance that presented clear physical image of interpretable deep neural networks, which makes it be possible for applying to analog computing device design, and pave the road to physical artificial intelligence.

Abstract

In this work, we generalize the reaction-diffusion equation in statistical physics, Schrödinger equation in quantum mechanics, Helmholtz equation in paraxial optics into the neural partial differential equations (NPDE), which can be considered as the fundamental equations in the field of artificial intelligence research. We take finite difference method to discretize NPDE for finding numerical solution, and the basic building blocks of deep neural network architecture, including multi-layer perceptron, convolutional neural network and recurrent neural networks, are generated. The learning strategies, such as Adaptive moment estimation, L-BFGS, pseudoinverse learning algorithms and partial differential equation constrained optimization, are also presented. We believe it is of significance that presented clear physical image of interpretable deep neural networks, which makes it be possible for applying to analog computing device design, and pave the road to physical artificial intelligence.

Partial Differential Equations is All You Need for Generating Neural Architectures -- A Theory for Physical Artificial Intelligence Systems

TL;DR

It is of significance that presented clear physical image of interpretable deep neural networks, which makes it be possible for applying to analog computing device design, and pave the road to physical artificial intelligence.

Abstract

In this work, we generalize the reaction-diffusion equation in statistical physics, Schrödinger equation in quantum mechanics, Helmholtz equation in paraxial optics into the neural partial differential equations (NPDE), which can be considered as the fundamental equations in the field of artificial intelligence research. We take finite difference method to discretize NPDE for finding numerical solution, and the basic building blocks of deep neural network architecture, including multi-layer perceptron, convolutional neural network and recurrent neural networks, are generated. The learning strategies, such as Adaptive moment estimation, L-BFGS, pseudoinverse learning algorithms and partial differential equation constrained optimization, are also presented. We believe it is of significance that presented clear physical image of interpretable deep neural networks, which makes it be possible for applying to analog computing device design, and pave the road to physical artificial intelligence.

Paper Structure

This paper contains 43 sections, 79 equations, 5 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. SLS Theory
  4. Turing's Equations
  5. Neural Partial Differential Equations
  6. Fisher's Equations
  7. Heat Equations
  8. Schrödinger Equations
  9. Paraxial Helmholtz Equations
  10. General NPDEs
  11. Generating Neural Architectures
  12. Two-model SLS
  13. Finite Difference Methods
  14. Convolutional Neural Networks
  15. 1D CNN
  16. ...and 28 more sections

Figures (5)

  • Figure 1: An SLS is constructed with two-model case: system generative model (System A) and system reduction model (System B).
  • Figure 2: Explanation of the elliptic operator for neural network based physics model described with RDEs.
  • Figure 3: Discretized diffusion equation as CNN. The time slice is equivalent to the layered structure along $z$ direction.
  • Figure 4: Multi-channel with full size convolutional kernel is equivalent to an MLP network structure.
  • Figure 5: Unfolded basic RNN. (Refer to Deeplearning)