Efficient Quantum Circuits for Machine Learning Activation Functions including Constant T-depth ReLU

TL;DR

This work tackles the challenge of embedding activation functions in fault-tolerant quantum circuits by prioritizing low $T$-depth. It delivers constant-$T$-depth realizations of ReLU ($4$) and Leaky ReLU ($8$) using Clifford+$T$ gates, and extends to other activations via Quantum Look-Up Tables (QLUT) with precision adjustable by qubit count, including 2D-grid connectivity considerations. Key contributions include the $n$-bit ReLU circuit with $2n-1$ qubits and $T$-depth $4$, a 2D-grid implementation maintaining constant $T$-depth with $ ext{Omega}( oot 2 floor{n})$ depth lower bound, and a Leaky ReLU circuit with $T$-depth $8$. The QLUT framework, using the SELECTSWAP network, enables nonarithmetic activations such as Sigmoid, Tanh, Swish, SoftMax, ELU, and GELU with tunable qubit/ancilla resources and depth, offering practical pathways for integrating activations into quantum neural networks.

Abstract

In recent years, Quantum Machine Learning (QML) has increasingly captured the interest of researchers. Among the components in this domain, activation functions hold a fundamental and indispensable role. Our research focuses on the development of activation functions quantum circuits for integration into fault-tolerant quantum computing architectures, with an emphasis on minimizing $T$-depth. Specifically, we present novel implementations of ReLU and leaky ReLU activation functions, achieving constant $T$-depths of 4 and 8, respectively. Leveraging quantum lookup tables, we extend our exploration to other activation functions such as the sigmoid. This approach enables us to customize precision and $T$-depth by adjusting the number of qubits, making our results more adaptable to various application scenarios. This study represents a significant advancement towards enhancing the practicality and application of quantum machine learning.

Figures (9)

• Figure 1: The quantum circuit implementing the $F_8$ gate.
• Figure 2: The quantum circuit implementing the ReLU function with input $\boldsymbol{x} \in \{0,1\}^5$ includes input qubits $\ket{x_i}$ and output qubits $\ket{0}_j$.
• Figure 3: The quantum circuit for the Toffoli gate.
• Figure 4: The quantum circuit for the four shared-control Toffoli gates.
• Figure 5: An illustration of qubit arrangement on a 2D grid, with $x_i$ representing input qubits and $a_j$ representing output qubits. The main idea of constructing the circuit is to partition the problem into four smaller ones, as illustrated in the figure.

Theorems & Definitions (13)

• Definition 3.1
• Lemma 3.2
• Theorem 3.3
• proof
• Theorem 3.4
• proof
• Theorem 3.5
• proof
• Theorem 3.6
• proof