Quantum noise modeling through Reinforcement Learning

TL;DR

This work introduces a machine learning approach to characterize the noise impacting a quantum chip and emulate it during simulations that leverages reinforcement learning (RL), offering increased flexibility in reproducing various noise models compared to conventional techniques such as randomized benchmarking or heuristic noise models.

Abstract

In the current era of quantum computing, robust and efficient tools are essential to bridge the gap between simulations and quantum hardware execution. In this work, we introduce a machine learning approach to characterize the noise impacting a quantum chip and emulate it during simulations. Our algorithm leverages reinforcement learning, offering increased flexibility in reproducing various noise models compared to conventional techniques such as randomized benchmarking or heuristic noise models. The effectiveness of the RL agent has been validated through simulations and testing on real superconducting qubits. Additionally, we provide practical use-case examples for the study of renowned quantum algorithms.

Figures (11)

• Figure 1: Example of a two-qubit quantum circuit (top) and its vector representation (bottom). A two-qubit circuit of depth three is represented as a tensor of size (2, 3, 8), where the first entry identifies the qubit, the second denotes the circuit moment, and the third specifies the type of gate or noise channel.
• Figure 2: Schematization of the policy neural network for the PPO algorithm. A common feature extractor ($\operatorname{FE}$), composed of a CNN, maps the QCR to a high-dimensional latent feature space. The resulting feature vector serves as input to both the actor policy ($\operatorname{A}\pi$) and the critic policy ($\operatorname{C}\pi$). The actor policy is responsible for selecting the best action for the agent, while the critic policy estimates the future reward for the action chosen by the actor.
• Figure 3: Training process of the RL algorithm. At each step the RL agent receives an observation of the environment, the quantum circuit representation of a non-noisy circuit. The agent policy performs an action that consists in putting any number of noise channels at a given circuit moment. At the end of the circuit the reward is computed to minimize the trace distance of the density matrix of the original noisy circuit and the reconstructed one.
• Figure 4: Average density matrix fidelity throughout the training process of the RL agent on single-qubit circuits (left) and three-qubit circuits (right). The metrics were evaluated on a dataset of 100 circuits for the single-qubit case and 800 circuits for the three-qubit case, using 80% for the training set and 20% for the test set. Shaded regions represent the standard deviation. In the single-qubit case, convergence of the training process was reached after approximately $4\times 10^5$ episodes, achieving an average fidelity of about $0.99$ on the test set. For the three-qubit case, convergence was reached after approximately $1.5\times 10^6$ episodes, achieving an average fidelity of about $0.98$ on the test set. No significant overfitting was observed in either case.
• Figure 6: Distribution of the noise channels parameters, depolarizing parameter (lambda) top and the amplitude damping parameter (gamma) bottom, inserted by the RL agent on a dataset of 100 random three-qubit Clifford circuits of depth 15. The average values of lambda and gamma are respectively $0.018$ and $0.029$, which are very close to the real parameters of the noise model used in the simulation ($0.02$ and $0.03$ respectively).