Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differentiable Quantum Computing for Large-scale Linear Control

Connor Clayton, Jiaqi Leng, Gengzhi Yang, Yi-Ling Qiao, Ming C. Lin, Xiaodi Wu

TL;DR

An end-to-end quantum algorithm for linear-quadratic control with provable speedups, based on a policy gradient method that incorporates a novel quantum subroutine for solving the matrix Lyapunov equation.

Abstract

As industrial models and designs grow increasingly complex, the demand for optimal control of large-scale dynamical systems has significantly increased. However, traditional methods for optimal control incur significant overhead as problem dimensions grow. In this paper, we introduce an end-to-end quantum algorithm for linear-quadratic control with provable speedups. Our algorithm, based on a policy gradient method, incorporates a novel quantum subroutine for solving the matrix Lyapunov equation. Specifically, we build a quantum-assisted differentiable simulator for efficient gradient estimation that is more accurate and robust than classical methods relying on stochastic approximation. Compared to the classical approaches, our method achieves a super-quadratic speedup. To the best of our knowledge, this is the first end-to-end quantum application to linear control problems with provable quantum advantage.

Differentiable Quantum Computing for Large-scale Linear Control

TL;DR

An end-to-end quantum algorithm for linear-quadratic control with provable speedups, based on a policy gradient method that incorporates a novel quantum subroutine for solving the matrix Lyapunov equation.

Abstract

As industrial models and designs grow increasingly complex, the demand for optimal control of large-scale dynamical systems has significantly increased. However, traditional methods for optimal control incur significant overhead as problem dimensions grow. In this paper, we introduce an end-to-end quantum algorithm for linear-quadratic control with provable speedups. Our algorithm, based on a policy gradient method, incorporates a novel quantum subroutine for solving the matrix Lyapunov equation. Specifically, we build a quantum-assisted differentiable simulator for efficient gradient estimation that is more accurate and robust than classical methods relying on stochastic approximation. Compared to the classical approaches, our method achieves a super-quadratic speedup. To the best of our knowledge, this is the first end-to-end quantum application to linear control problems with provable quantum advantage.

Paper Structure

This paper contains 39 sections, 35 theorems, 128 equations, 4 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notation.
  3. Problem Formulation
  4. Main Contributions
  5. Related Work
  6. Model-based linear-quadratic control.
  7. Model-free linear-quadratic control.
  8. Differentiable physics and quantum computing.
  9. Quantum reinforcement learning.
  10. Preliminaries
  11. Introduction to Quantum Computing
  12. Policy Gradient for LQR
  13. Quantum Data Structure
  14. Quantum Simulation of Linear Dynamics
  15. Quantum Algorithm for the Lyapunov Equation
  16. ...and 24 more sections

Key Result

Lemma 1

Let $K \in \mathbb{R}^{m \times n}$. There exists a data structure to store $K$ with the following properties: (1) the size of the data structure is $\mathcal{O}\left(mn\log^2(mn)\right)$, (2) the time to store a new entry $(i, j, \hat{K}_{i,j})$ is $\mathcal{O}\left(\log^2(mn)\right)$, and (3) for

Figures (4)

  • Figure 1: Differentiable quantum computing for linear control.
  • Figure 2: Numerical Results on Convergence. Following the mass-spring-damper setup in mohammadi2021convergence, our policy gradient descent algorithm converges much faster than mohammadi2021convergence.
  • Figure 3: Numerical Results on Convergence. In the aircraft control problem, our policy gradient descent algorithm converges much faster than classic method mohammadi2021convergence.
  • Figure 4: Relative Error. We scale the size of a mass-spring system and our method consistently gets smaller relative error compared to mohammadi2021convergence.

Theorems & Definitions (76)

  • Definition 1
  • Definition 2
  • Definition 3: Block-encoding
  • Lemma 1
  • proof
  • Theorem 2: Informal version of \ref{['thm:main-block-encode-mat-exp']}
  • Lemma 3: Numerical integration
  • proof
  • Definition 4: Select oracle
  • Theorem 4
  • ...and 66 more