Benchmarking Optimization Algorithms for Automated Calibration of Quantum Devices

TL;DR

The paper investigates optimization algorithms for automated calibration of quantum devices in the presence of noise and drift, using a simulated single-qubit setup with DRAG and PWC pulse representations. It compares several gradient-free optimizers, highlighting CMA-ES as the most effective across low- and high-dimensional calibration tasks and underscoring the critical role of the loss function in achieving high fidelities. The study demonstrates CMA-ES’s robustness to noise and local minima, while also discussing practical strategies like hybrid optimization and hyperparameter tuning. These findings have practical implications for accelerating QPU bring-up and tune-up, and point to avenues for developing even better, potentially quantum-tailored calibration algorithms.

Abstract

We present the results of a comprehensive study of optimization algorithms for the calibration of quantum devices. As part of our ongoing efforts to automate bring-up, tune-up, and system identification procedures, we investigate a broad range of optimizers within a simulated environment designed to closely mimic the challenges of real-world experimental conditions. Our benchmark includes widely used algorithms such as Nelder-Mead and the state-of-the-art Covariance Matrix Adaptation Evolution Strategy (CMA-ES). We evaluate performance in both low-dimensional settings, representing simple pulse shapes used in current optimal control protocols with a limited number of parameters, and high-dimensional regimes, which reflect the demands of complex control pulses with many parameters. Based on our findings, we recommend the CMA-ES algorithm and provide empirical evidence for its superior performance across all tested scenarios.

Figures (5)

• Figure 1: Simulated landscape of loss function specified in Eq. (\ref{['eq:loss_infid_orbit']}) for a single ORBIT sequence $n=1$ and $l = 80$ for a DRAG pulse. The typical DRAG minimum is clearly visible. The landscape exhibits multiple local minima, which can hinder the calibration process, as optimizers may converge to sub optimal regions. Averaging over a sufficiently large number of ORBIT sequences helps smooth out these features and improves the overall convergence behavior of the optimizer.
• Figure 2: In-phase and quadrature components of the DRAG and PWC pulses for an $R_x(\theta = \frac{\pi}{2})$ gate. The PWC pulse exhibits the characteristic discretized step structure, where each step value is independently optimized and forms part of the 82-dimensional optimization space. At the start of the optimization, the step heights are randomized to simulate an uninformed initial guess.
• Figure 3: Hyperparameter optimization using the CMA-ES algorithm. Panel \ref{['fig:drag_cmaes_hyperparam_benchmark']} shows the progression of the overall optimization score, which combines final infidelity and convergence rate. Panel \ref{['fig:drag_cmaes_hyperparam_scatter']} shows the history of evaluated hyperparameter values, illustrating how some configurations degrade performance.
• Figure 4: Comparison of 120 CMA-ES optimization runs using optimized hyperparameters versus randomly detuned hyperparameters. The figure shows the mean and median performance of the instances. The results demonstrate that CMA-ES performs better and converges with fewer function evaluations when hyperparameters are optimized.
• Figure 5: Benchmarking results for the DRAG and PWC pulse simulations. The x-axis shows the number of function evaluations, and the y-axis displays the corresponding loss function value for each parameter setting. Colored lines represent the performance of different optimization algorithms. Figures \ref{['fig:drag_mean_comp']} and \ref{['fig:pwc_mean_comp']} show the mean values across 120 simulations for each optimizer. Figures \ref{['fig:drag_median_comp']} and \ref{['fig:pwc_median_comp']} show the corresponding medians. CMA-ES achieves the lowest infidelity in all cases.