Effects of noise on the overparametrization of quantum neural networks

TL;DR

This work proves that as the magnitude of noise increases all the eigenvalues of the QFIM become exponentially suppressed, indicating that the state becomes insensitive to any change in the parameters, and implies that current QNN capacity measures are ill-defined when hardware noise is present.

Abstract

Overparametrization is one of the most surprising and notorious phenomena in machine learning. Recently, there have been several efforts to study if, and how, Quantum Neural Networks (QNNs) acting in the absence of hardware noise can be overparametrized. In particular, it has been proposed that a QNN can be defined as overparametrized if it has enough parameters to explore all available directions in state space. That is, if the rank of the Quantum Fisher Information Matrix (QFIM) for the QNN's output state is saturated. Here, we explore how the presence of noise affects the overparametrization phenomenon. Our results show that noise can "turn on" previously-zero eigenvalues of the QFIM. This enables the parametrized state to explore directions that were otherwise inaccessible, thus potentially turning an overparametrized QNN into an underparametrized one. For small noise levels, the QNN is quasi-overparametrized, as large eigenvalues coexists with small ones. Then, we prove that as the magnitude of noise increases all the eigenvalues of the QFIM become exponentially suppressed, indicating that the state becomes insensitive to any change in the parameters. As such, there is a pull-and-tug effect where noise can enable new directions, but also suppress the sensitivity to parameter updates. Finally, our results imply that current QNN capacity measures are ill-defined when hardware noise is present.

Figures (11)

• Figure 1: Schematic diagram of our main results. Consider the task of implementing a QNN, i.e., a parametrized unitary channel on a quantum computer. As shown in larocca2021theory, the overparametrization phenomenon is defined as the QNN having enough parameters to explore all relevant directions in state space. a) For certain ansatzes the QNN can be efficiently overparametrized with few parameters, as there only exists a small number of available directions in state space. Moreover, for (most) such directions, changes in the parameter values usually translate into changes in state space. b) When the quantum device is faulty, quantum noise will act throughout the computation. In this work, we explore how hardware noise modifies the overparametrization phenomenon. Our results show that quantum noise can enable additional directions in state space. However, we also find that as the noise probability increases, the system becomes more and more insensitive to variations in the parameters.
• Figure 2: QFIM and directions in state space. Let $U(\boldsymbol{\theta})\in e^{\mathfrak{g}}$ be a parametrized unitary and $\ket{\psi}$ some pure state, so that $\ket{\psi(\boldsymbol{\theta})}=U(\boldsymbol{\theta})\ket{\psi}$. Here we schematically show that the eigenvalues and eigenvectors of the QFIM, $F(\ket{\psi(\boldsymbol{\theta})})$, inform how changes in parameter space translate into changes in state space. In particular, when modifying $\boldsymbol{\theta}$ following the eigenvectors of the QFIM, the state $\ket{\psi(\boldsymbol{\theta})}$ explores the corresponding available direction in the tangent space $\mathfrak{g}\ket{\psi(\boldsymbol{\theta})}$. Additionally, the magnitude of the QFIM eigenvalues determines the sensitivity of the state to a change along an eigenvector direction meyer2021fisher. As such, a large eigenvalue means that it is "easy" to nudge the state in state space, while small eigenvalues indicate that the state is insensitive to parameter changes in the direction of the associated eigenvector.
• Figure 3: Noiseless and noisy quantum circuits. a) Noiseless quantum circuit consisting of parametrized unitary channels $\mathcal{C}^m_{\theta_m}$. b) Noisy quantum circuit where unital Pauli noise channels $\mathcal{N}_m$ are interleaved with the unitary channels.
• Figure 4: Single-qubit toy model examples. a) We consider the case where the single qubit state of Eq. \ref{['eq:si-state']} is sent through a noiseless QNN with four parameters as in Eq. \ref{['eq:si_qnn']}. We plot in the Bloch sphere the three trajectories defined by $\boldsymbol{\theta}_1$, $\boldsymbol{\theta}_2$ and $\boldsymbol{\theta}_3$. b) We consider the case where the single qubit state of Eq. \ref{['eq:si-state']} is sent through a noisy QNN with four parameters, as in Eq. \ref{['eq:si-noisy-qnn']}. Here, bit-flip noise channels act before and after every gate with probability $p=0.1$. We plot in the Bloch sphere the three trajectories defined by $\boldsymbol{\theta}_1$, $\boldsymbol{\theta}_2$ and $\boldsymbol{\theta}_3$. The action of the unitary gates is marked in blue, whereas the action of the noise channels is marked in red.
• Figure 5: State space trajectories following perturbations along QFIM eigendirections. a) We consider that the single qubit state of Eq. \ref{['eq:si-state']} is sent through a noiseless QNN as in Eq. \ref{['eq:si_qnn']}, with parameters $\boldsymbol{\theta}_3$. Here we show within the Bloch sphere how the state $\rho_{\boldsymbol{\theta}}$ changes when the parameters are varied following the directions given by three eigenvectors of the QFIM $F(\rho_{\boldsymbol{\theta}})$. Two such directions are associated with the two non-zero eigenvalues (blue and red curves) and with a zero eigenvalue (green, non-visible, curve). b) We consider that the single qubit state of Eq. \ref{['eq:si-state']} is sent through a noisy QNN as in Eq. \ref{['eq:si-noisy-qnn']}, with parameters $\boldsymbol{\theta}_3$. Here we show within the Bloch sphere how the state $\rho_{\boldsymbol{\theta}}$ changes when the parameters are varied following the directions given by the three eigenvectors of the QFIM $F(\widetilde{\rho}_{\boldsymbol{\theta}})$ with associated non-zero eigenvalues (blue, red, and green curves).

Theorems & Definitions (17)

• Definition 1: Dynamical Lie Algebra
• Definition 2: Overparametrization
• Definition 3: Unital Pauli channel
• Theorem 1
• Theorem 2
• Theorem 3
• Theorem 4
• proof
• proof
• proof