Random-Matrix-Induced Simplicity Bias in Over-parameterized Variational Quantum Circuits

TL;DR

This work explains why increasing expressivity via unstructured over-parameterization can harm learnability in variational quantum circuits by revealing a Haar-like universality that concentrates outputs and gradients, causing a simplicity bias. Using random-matrix theory, it proves that sufficiently expressive, unstructured VQCs collapse to near-constant functions for typical parameter choices, with gradients vanishing exponentially in the number of qubits. It then shows that tensor-structured VQCs, through bounded tensor rank or bond dimension, break this universality, preserving output variability and nondegenerate gradients, thereby restoring a nontrivial learnable hypothesis class. The results unify barren plateaus, expressivity limits, and generalization collapse as manifestations of unitary-ensemble geometry and offer a design principle: structure the circuit to avoid approximate unitary designs to maintain learnability. Practically, tensor-network-inspired architectures emerge as principled mitigations for over-parameterized quantum models, guiding scalable and trainable quantum algorithms.

Abstract

Over-parameterization is commonly used to increase the expressivity of variational quantum circuits (VQCs), yet deeper and more highly parameterized circuits often exhibit poor trainability and limited generalization. In this work, we provide a theoretical explanation for this phenomenon from a function-class perspective. We show that sufficiently expressive, unstructured variational ansatze enter a Haar-like universality class in which both observable expectation values and parameter gradients concentrate exponentially with system size. As a consequence, the hypothesis class induced by such circuits collapses with high probability to a narrow family of near-constant functions, a phenomenon we term simplicity bias, with barren plateaus arising as a consequence rather than the root cause. Using tools from random matrix theory and concentration of measure, we rigorously characterize this universality class and establish uniform hypothesis-class collapse over finite datasets. We further show that this collapse is not unavoidable: tensor-structured VQCs, including tensor-network-based and tensor-hypernetwork parameterizations, lie outside the Haar-like universality class. By restricting the accessible unitary ensemble through bounded tensor rank or bond dimension, these architectures prevent concentration of measure, preserve output variability for local observables, and retain non-degenerate gradient signals even in over-parameterized regimes. Together, our results unify barren plateaus, expressivity limits, and generalization collapse under a single structural mechanism rooted in random-matrix universality, highlighting the central role of architectural inductive bias in variational quantum algorithms.

Figures (4)

• Figure 1: Schematic illustration of simplicity bias and its mitigation in VQCs. Top: In sufficiently expressive, unstructured variational quantum circuits, the induced unitary ensemble approaches a random-matrix universality class, leading to concentration of observable expectation values and gradients. As a result, the hypothesis class collapses to near-constant functions, giving rise to simplicity bias and barren plateaus. Bottom: Tensor-structured variational quantum circuits restrict the accessible unitary manifold through bounded tensor rank or bond dimension, preventing concentration of measure. This structural constraint preserves output variability and informative gradients, thereby mitigating simplicity bias even in over-parameterized regimes.
• Figure 2: Schematic of a VQC considered in this work. A VQC consists of an input quantum state initialized as $\vert 0 \rangle^{\otimes n}$, followed by a data-encoding unitary $U(\textbf{x})$, a parameterized variational ansätze $W(\boldsymbol{\theta})$, and a measurement of a Hermitian observable $O$, yielding the scalar output $f_{\boldsymbol{\theta}}(\textbf{x})$. The expressive and statistical properties of the induced function family $\{f_{\boldsymbol{\theta}}\}$ depend on the structure of the variational ansätze $W(\boldsymbol{\theta})$.
• Figure 3: Tensor-structured VQC architectures. (a) TN-VQC: A tensor network module transforms the classical input $\textbf{x}$ into a lower-dimensional feature $\mathcal{T}(\textbf{x}; \boldsymbol{\phi})$, which is further converted into quantum state $U\circ \mathcal{T}(\textbf{x}; \boldsymbol{\phi})$ via the encoding unitary $\mathcal{T}(\textbf{x}; \boldsymbol{\phi})$. The encoded quantum state is then processed by a VQC $W(\boldsymbol{\theta})$ and measured via an observable $O$ to produce the output $f_{\boldsymbol{\theta}}(\textbf{x})$. Both the encoding parameters $\boldsymbol{\phi}$ and the circuit parameters $\boldsymbol{\theta}$ are updated through gradient-based optimization. (b) TensorHyper-VQC: A tensor-network acts as a hypernetwork that generates the variational circuit parameters $\boldsymbol{\theta}$ directly through $\mathcal{T}(\boldsymbol{\sigma}; \boldsymbol{\phi})$ by using a Gaussian random vector $\boldsymbol{\sigma}$. The data encoding is fixed, while the tensor network induces structured correlations among the parameters of $W(\boldsymbol{\theta})$. In both architectures, the tensor-network structure constrains the accessible unitary ensemble, breaking Haar-like typicality and mitigating random-matrix-induced simplicity bias.
• Figure 4: Finite-sample input variance and breakdown of typicality. Empirical variance of the model output $\operatorname{Var}_{\textbf{x} \sim \mathcal{D}_{m}}[f_{\boldsymbol{\theta}}(\textbf{x})]$ evaluated on datasets of size $m=2^3, ..., 2^8$ for a naive VQC, a TN-VQC, and a TensorHyper-VQC. Values are averaged over random initializations, with error bars denoting one standard deviation. Tensor-structured architectures exhibit stable, non-vanishing across dataset sizes, in contrast to the strongly concentrated behavior of the unstructured VQC.

Theorems & Definitions (12)

• Definition 1: Haar-random
• Theorem 1: Output concentration
• Theorem 2: Gradient concentration
• Definition 2: Simplicity bias
• Corollary 1: Uniform hypothesis-class collapse over finite datasets
• Proposition 1: Failure of Approximate Unitary Designs
• Lemma 1
• Proposition 2: Anti-concentration
• Definition 3: Non-trivial hypothesis class
• Theorem 3: Tensor-structured VQC mitigates simplicity bias