The effect of the number of parameters and the number of local feature patches on loss landscapes in distributed quantum neural networks

TL;DR

This work examines how the architecture of distributed QNNs using local patches affects loss landscape geometry when classifying classical data. By applying Hessian analysis and loss-landscape visualization, it shows that increasing the number of parameters sharpens minima, while increasing the number of local patches flattens them, yielding improved stability and generalization. The results suggest that a patch-based distributed QNN approach can mitigate optimization challenges in quantum machine learning on tasks like MNIST image classification. This has practical implications for designing trainable, scalable QML models that leverage local feature patches to enhance trainability and performance on classical data tasks.

Abstract

Quantum neural networks hold promise for tackling computationally challenging tasks that are intractable for classical computers. However, their practical application is hindered by significant optimization challenges, arising from complex loss landscapes characterized by barren plateaus and numerous local minima. These problems become more severe as the number of parameters or qubits increases, hampering effective training. To mitigate these optimization challenges, particularly for quantum machine learning applied to classical data, we employ an approach of distributing overlapping local patches across multiple quantum neural networks, processing each patch with an independent quantum neural network, and aggregating their outputs for prediction. In this study, we investigate how the number of parameters and patches affects the loss landscape geometry of this distributed quantum neural network architecture via Hessian analysis and loss landscape visualization. Our results confirm that increasing the number of parameters tends to lead to deeper and sharper loss landscapes. Crucially, we demonstrate that increasing the number of patches significantly reduces the largest Hessian eigenvalue at minima. This finding suggests that our distributed patch approach acts as a form of implicit regularization, promoting optimization stability and potentially enhancing generalization. Our study provides valuable insights into optimization challenges and highlights that the distributed patch approach is a promising strategy for developing more trainable and practical quantum machine learning models for classical data tasks.

Figures (5)

• Figure 1: Overview of the model used in our numerical experiments: First, an input image is partitioned into smaller patches. Each patch is added with positional bias terms $b'_{h,w}$, and processed by one of $n_\text{qc}$ different QNNs, parameterized by $\bm{\Phi}_p$. Then, the expectation values from the QNNs are averaged. This averaged vector $\overline{\bm{y}}_i$ is scaled and passed through the Softmax function to obtain classification predictions. Finally, using these predictions, the cross-entropy loss function is evaluated, and all trainable parameters, including parameters $\bm{\Phi}_0,\ldots,\bm{\Phi}_{n_\text{qc}-1}$ and bias terms $\bm{b}'_{h,w}$, are optimized to minimize this loss. This figure is adapted from Ref. kawase2024distributed, CC BY 4.0.
• Figure 2: This figure illustrates the process of obtaining patches from an image. First, we extract a $P\times P$ patch from the top-left corner. Then, we shift the window $D$ pixels to the right to extract the next patch. This horizontal extraction is repeated across the row until $L$ patches are obtained. After completing one row, we shift the starting position $D$ pixels downward. We then repeat the same horizontal extraction process for this new row. This procedure continues row by row until a total of $L^2$ patches have been extracted.
• Figure 3: Visualization of training loss landscapes and optimization trajectories for each model: This figure illustrates the training loss landscapes and the optimization trajectories, projected onto the 2D plane defined by the first two principal components of the model parameters (the first and the second principal components are denoted by PC1 and PC2, respectively). The optimization trajectory starts at the position marked by a red cross and ends at the dark blue cross. The orange cross marks the position of the minimum training loss during optimization. The contour lines depict levels of the loss function at intervals of $0.5$.
• Figure 4: Figure \ref{['fig:qc_overview']} shows the overview of a quantum circuit we used in our numerical experiments. Note that $d$ and $n$ represent the number of repetitions of the entangling block $V_1$ and the number of qubits. In our numerical experiments, we set $n=8$, and $d=50, 100, 150$ or $200$.
• Figure 5: These figures show the training and test losses at the end of each epoch. From these figures, we can see that the losses periodically increase or decrease, because we used Adam with a cosine annealing scheduler in our numerical experiments, as we mentioned in Section \ref{['subsec:settings']}.