Quantum-Inspired Weight-Constrained Neural Network: Reducing Variable Numbers by 100x Compared to Standard Neural Networks

TL;DR

A dropout method is developed to enhance the robustness of quantum machine learning models, which are highly susceptible to adversarial attacks, and a novel approach to reduce the complexity of large classical neural networks is offered, addressing a critical challenge in machine learning.

Abstract

Although quantum machine learning has shown great promise, the practical application of quantum computers remains constrained in the noisy intermediate-scale quantum era. To take advantage of quantum machine learning, we investigate the underlying mathematical principles of these quantum models and find that the quantum neural network with amplitude encoding is equivalent to a weight-constrained neural network. Motived by this discovery, we develop a classical weight-constrained neural network. We find that this approach can reduce the number of variables in a classical neural network by a factor of 135 while preserving its accuracy. In addition, we develop a dropout method to enhance the robustness of quantum machine learning models, which are highly susceptible to adversarial attacks. This technique can also be applied to improve the adversarial robustness of the classical weight-constrained neural network, which is essential for industry applications, such as self-driving vehicles. Our work offers a novel approach to reduce the complexity of large classical neural networks, addressing a critical challenge in machine learning.

Figures (12)

• Figure 1: Hybrid quantum-classical neural network. (a) A generic quantum circuit with angle encoding. (b) A generic quantum circuit with amplitude encoding.
• Figure 2: Panel (a) and panel (b) plot the distribution $p(z)$ of $z$, defined as $z=\sum_i w_i x_i$, where $w$ denotes the weight, and $x$ denotes the input features. (c) The inverse of Kullback-Leibler divergence $D_\text{KL}^{-1}$ of the distribution $p(z)$ as a function of $N$ at $r=4$, where $N$ denotes the number of variables. (d) $D_\text{KL}^{-1}$ of the distribution $p(z)$ as a function of $r$ at $N=16$.
• Figure 3: Training results of different neural networks. (a) A summary of the accuracy (ACC) of these five models for MNIST and FMNIST datasets. (b) The architecture of the hybrid quantum-classical neuron network (HNN) and the training results on the MNIST and Fashion MNIST (FMNIST) datasets. (c) The fully connected neuron network (FNN) architecture and the training results on the MNIST and FMNIST datasets. (d) The weight-constrained FNN architecture and the training results on the MNIST and FMNIST datasets. (e) The convolutional neural network (CNN) architecture and the training results on the MNIST and FMNIST datasets. (f) The architecture of the weight-constrained CNN and the training results on the MNIST and FMNIST datasets.
• Figure 4: The accuracy (ACC) of the weight-constrained fully connected neural network (FNN) and the weight-constrained convolutional neural network (CNN) for different values of $N$ and $r$. Here, $N$ and $r$ represent variables in the combination formula $C(N,r)$. Panel (a) and panel (b) plot the results of the MNIST and the FMNIST datasets. Panel (c) and panel (d) plot the results of the CIFAR and the TRAFFIC datasets. The red dashed line and shade region represent the optimal value and the $\pm1$ standard deviation interval of the standard network, respectively.
• Figure 5: Test accuracy (ACC) of neural networks with different weight parameterizations. (a) MNIST and (b) FMNIST results for fully-connected neural networks (FNNs). (c) CIFAR and (d) TRAFFIC dataset results for convolutional neural networks (CNNs). For each panel, performance is shown for a standard network, our proposed weight-constrained method, and a low-rank method with varying $m$. Markers indicate the mean ACC over 16 simulations (circles) and the optimal result (diamonds).