Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Automatic Implementation of Neural Networks through Reaction Networks--Part II: Error Analysis

Yuzhen Fan, Xiaoyu Zhang, Chuanhou Gao, Denis Dochain

TL;DR

A systematic framework for error analysis of the designed biochemical fully connected neural network (BFCNN) is established and the formula for the total error upper bound about the iteration number is derived.

Abstract

This paired article aims to develop an automated and programmable biochemical fully connected neural network (BFCNN) with solid theoretical support. In Part I, a concrete design for BFCNN is presented, along with the validation of the effectiveness and exponential convergence of computational modules. In this article, we establish the framework for specifying the realization errors by monitoring the errors generated from approaching equilibrium points in individual modules, as well as their vertical propagation from upstream modules and horizontal accumulation from previous iterations. Ultimately, we derive the general error upper bound formula for any iteration and illustrate its exponential convergence order with respect to the phase length of the utilized chemical oscillator. The numerical experiments, based on the classification examples, reveal the tendency of total errors related to both the phase length and iteration number.

Automatic Implementation of Neural Networks through Reaction Networks--Part II: Error Analysis

TL;DR

A systematic framework for error analysis of the designed biochemical fully connected neural network (BFCNN) is established and the formula for the total error upper bound about the iteration number is derived.

Abstract

This paired article aims to develop an automated and programmable biochemical fully connected neural network (BFCNN) with solid theoretical support. In Part I, a concrete design for BFCNN is presented, along with the validation of the effectiveness and exponential convergence of computational modules. In this article, we establish the framework for specifying the realization errors by monitoring the errors generated from approaching equilibrium points in individual modules, as well as their vertical propagation from upstream modules and horizontal accumulation from previous iterations. Ultimately, we derive the general error upper bound formula for any iteration and illustrate its exponential convergence order with respect to the phase length of the utilized chemical oscillator. The numerical experiments, based on the classification examples, reveal the tendency of total errors related to both the phase length and iteration number.
Paper Structure (21 sections, 15 theorems, 107 equations, 7 figures)

This paper contains 21 sections, 15 theorems, 107 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Chemical Reaction Network
  4. Fully Connected Neural Network
  5. Feedforward Propagation
  6. Backward Propagation
  7. Composition of Errors
  8. Current error $\bar{\epsilon}_i^m$
  9. Accumulative error $\tilde{\epsilon}^m_i$
  10. Error Estimation in the First Iteration
  11. Assignment Module Error
  12. Feedforward Module Error
  13. lwsBCRN
  14. sigBCRN
  15. Error in the Output Layer
  16. ...and 6 more sections

Key Result

Proposition 1

Let $s=(\chi_{.{1}},...,\chi_{.\tilde{p}})^{\top}$, $\tilde{c}=\mathds{O}_{p \times \tilde{p}}$, $c = (e_{1+\tilde{p}},\cdots, e_{2\tilde{p}})^\top$ where $e_l \in \mathbb{R}^{1 \times p}$ is a unit row vector with the $l$th element to be 1 and $I_l\triangleq\{l,\tilde{p}+l,...,p-2\tilde{p}+l\}$, th where $k>0$ is the rate constant in $\mathcal{M}^a_2, \mathcal{M}^a_3$.

Figures (7)

  • Figure 1: The propagation of the current error. The vertical thin black arrow denotes the vertical propagation, and the horizontal thick arrow describes the horizontal propagation.
  • Figure 2: Schema of the propagation of error through initial concentrations from the $m-1$th to the $m$th iteration in the Assignment module.
  • Figure 3: Schema of the propagation of error via initial concentrations and parameters from the $m-1$th to the $m$th iteration in the Feedforward module. The top two blocks represent the hidden layer neurons, and the bottom one denotes the output neuron.
  • Figure 4: Schema of the propagation of error via initial concentrations and parameters from the $m-1$th to the $m$th iteration in the Preceding Calculations module.
  • Figure 5: Schema of the propagation of error via initial concentrations and parameters from the $m-1$th to the $m$th iteration in the Learning Module.
  • ...and 2 more figures

Theorems & Definitions (34)

  • Definition 1: CRN
  • Proposition 1
  • proof
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • Remark 1
  • ...and 24 more