Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Noise-robust chemical reaction networks training artificial neural networks

Sunghwa Kang, Jinsu Kim

TL;DR

This work presents a noise-robust CRN for full NN computation, including backpropagation, paving the way for more stable and efficient biochemical computing systems.

Abstract

Artificial neural networks (NNs) can be implemented using chemical reaction networks (CRNs), where the concentrations of species act as inputs and outputs. In such biochemical computing, noise-robust computing is crucial due to the intrinsic and extrinsic noise present in chemical reactions. Previously suggested CRNs for feed-forward networks often utilized the rectified linear unit (ReLU) or discrete activation functions. However, one concern in this case is the discontinuities of the derivatives of those non-smooth functions, which can cause significant noise disruption during backpropagation. In this study, we propose a CRN that performs both feed-forward and training processes using smooth activation functions to avoid discontinuities in the backpropagation. All reactions occur in a single pot, and the reactions for training are bimolecular. Our case studies on XOR, Iris, MNIST datasets, and a non-linear regression model demonstrate that computation via the CRN (i) maintains accuracy despite noise in the reaction rates and the concentration of species and (ii) is insensitive to the choice of the running time and the magnitude of the noise in comparison to NNs with a non-smooth activation function. This work presents a noise-robust CRN for full NN computation, including backpropagation, paving the way for more stable and efficient biochemical computing systems.

Noise-robust chemical reaction networks training artificial neural networks

TL;DR

This work presents a noise-robust CRN for full NN computation, including backpropagation, paving the way for more stable and efficient biochemical computing systems.

Abstract

Artificial neural networks (NNs) can be implemented using chemical reaction networks (CRNs), where the concentrations of species act as inputs and outputs. In such biochemical computing, noise-robust computing is crucial due to the intrinsic and extrinsic noise present in chemical reactions. Previously suggested CRNs for feed-forward networks often utilized the rectified linear unit (ReLU) or discrete activation functions. However, one concern in this case is the discontinuities of the derivatives of those non-smooth functions, which can cause significant noise disruption during backpropagation. In this study, we propose a CRN that performs both feed-forward and training processes using smooth activation functions to avoid discontinuities in the backpropagation. All reactions occur in a single pot, and the reactions for training are bimolecular. Our case studies on XOR, Iris, MNIST datasets, and a non-linear regression model demonstrate that computation via the CRN (i) maintains accuracy despite noise in the reaction rates and the concentration of species and (ii) is insensitive to the choice of the running time and the magnitude of the noise in comparison to NNs with a non-smooth activation function. This work presents a noise-robust CRN for full NN computation, including backpropagation, paving the way for more stable and efficient biochemical computing systems.

Paper Structure

This paper contains 25 sections, 4 theorems, 50 equations, 11 figures, 4 tables.

Table of Contents

  1. Introduction
  2. The key concepts
  3. Chemical reaction networks
  4. Artificial neural networks
  5. Gradient of the loss function and derivatives of smooth activation functions.
  6. CRNs for training
  7. Backward CRN
  8. Update network
  9. One-pot computing and convergence
  10. An example of the the backward and the update network
  11. Applications to data sets
  12. Robustness to noisy reaction rates
  13. Noisy gradient descent processes induced by fluctuations in the CRN
  14. Comparison to CRNs implementing NNs with non-smooth activation functions
  15. Sensitivity to noise level and the running time
  16. ...and 10 more sections

Key Result

Proposition 5.1

Under the above setting, there exists a finite random variable $\Theta$ such that for each $\ell$ and for $i$, $\sup_{t\in [0,T]}|x^\ell_i(t;\epsilon)-x^\ell_i(t)| \le \epsilon \Theta$, where

Figures (11)

  • Figure 1: Abstract of our work.
  • Figure 2: An example of chemical reaction networks
  • Figure 3: A. The graph of the smooth ReLU with different smoothing parameters $h$. B. The graph of the leaky ReLU. C. An example of neural networks.
  • Figure 4: Overall structure of the CRN implementing the backpropagation of a two-layered NN. The green boxes indicate a CRN that calculates the designated arithmetic operations. Each green box has two input species indicated by the incoming arrows. The outgoing arrows indicate the output species. The full lists of reactions for the backward and the update CRNs are provided in Section \ref{['sec: full backward and update']}.
  • Figure 5: A and B. A schematic description of the convergence of the backward and the update networks without noise (A) and with noise (B). C. The time evolutions of the species. The vertical dotted lines indicate the times when the input of the NN is changed randomly. In particular, the concentration $w^1_{11}(t)$ of the species $W^1_{11}$ corresponding to a weight parameter evolves slowly.
  • ...and 6 more figures

Theorems & Definitions (11)

  • Definition 2.1
  • Definition 2.2
  • Remark 2.1
  • Remark 3.1
  • Proposition 5.1
  • Proposition 5.2
  • Remark 5.1
  • Lemma A.1
  • proof
  • Lemma B.1
  • ...and 1 more