Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantum Algorithm for Sparse Online Learning with Truncated Gradient Descent

Debbie Lim, Yixian Qiu, Patrick Rebentrost, Qisheng Wang

TL;DR

A quantum sparse online learning algorithm for logistic regression, the SVM, and least squares is developed, showing that a quadratic speedup in the time complexity with respect to the dimension of the problem is achievable, while maintaining a regret of $O(1/\sqrt{T})$, where $T$ is the number of iterations.

Abstract

Logistic regression, the Support Vector Machine (SVM), and least squares are well-studied methods in the statistical and computer science community, with various practical applications. High-dimensional data arriving on a real-time basis makes the design of online learning algorithms that produce sparse solutions essential. The seminal work of \hyperlink{cite.langford2009sparse}{Langford, Li, and Zhang (2009)} developed a method to obtain sparsity via truncated gradient descent, showing a near-optimal online regret bound. Based on this method, we develop a quantum sparse online learning algorithm for logistic regression, the SVM, and least squares. Given efficient quantum access to the inputs, we show that a quadratic speedup in the time complexity with respect to the dimension of the problem is achievable, while maintaining a regret of $O(1/\sqrt{T})$, where $T$ is the number of iterations.

Quantum Algorithm for Sparse Online Learning with Truncated Gradient Descent

TL;DR

A quantum sparse online learning algorithm for logistic regression, the SVM, and least squares is developed, showing that a quadratic speedup in the time complexity with respect to the dimension of the problem is achievable, while maintaining a regret of , where is the number of iterations.

Abstract

Logistic regression, the Support Vector Machine (SVM), and least squares are well-studied methods in the statistical and computer science community, with various practical applications. High-dimensional data arriving on a real-time basis makes the design of online learning algorithms that produce sparse solutions essential. The seminal work of \hyperlink{cite.langford2009sparse}{Langford, Li, and Zhang (2009)} developed a method to obtain sparsity via truncated gradient descent, showing a near-optimal online regret bound. Based on this method, we develop a quantum sparse online learning algorithm for logistic regression, the SVM, and least squares. Given efficient quantum access to the inputs, we show that a quadratic speedup in the time complexity with respect to the dimension of the problem is achievable, while maintaining a regret of , where is the number of iterations.

Paper Structure

This paper contains 30 sections, 8 theorems, 34 equations, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. The online learning framework.
  3. Main contribution.
  4. Techniques.
  5. Applications.
  6. Related Work
  7. Preliminaries
  8. Notations.
  9. Quantum computational model.
  10. Truncated gradient descent.
  11. Quantum Algorithm
  12. Quantum input and output model
  13. Quantum subroutines
  14. Quantum algorithm for sparse online learning
  15. Convergence and Time Complexity Analysis
  16. ...and 15 more sections

Key Result

Lemma 4.1

Let $a, b, x\in\{0, 1\}^n$ and $c, z\in\{0, 1\}$. The values of $s_{comp}$ and $s'_{comp}$ depend on the type of circuit architecture used for the comparators gidney2018halvingcuccaro2004new. Nevertheless, these are in general $O(n)$. We say that we have access to a minmax oracle if we have access to a unitary $U_{\text{minmax}}$ that performs the using $O(n)$ number of Toffoli gates.

Theorems & Definitions (9)

  • Lemma 4.1: Minmax and Between oracles luongo2024measurement
  • Lemma 4.2: Truncation unitary
  • Lemma 4.3: Quantum norm estimation and state preparation van2019quantumbrassard2002quantumhamoudi2019quantumli2019sublinearrebentrost2021quantum
  • Lemma 4.4: Quantum inner product estimation yang2023quantum
  • Lemma 4.5
  • Theorem 5.1: Regret for online logistic regression
  • Theorem 5.2: Regret for online hinge loss
  • Theorem 5.3: Regret for online least squares
  • Remark B.1