Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Hybrid Quantum-Classical Approach based on the Hadamard Transform for the Convolutional Layer

Hongyi Pan, Xin Zhu, Salih Atici, Ahmet Enis Cetin

TL;DR

The paper tackles the computational burden of convolutional layers by introducing a Hadamard Transform (HT)-based HT-perceptron layer for hybrid quantum-classical neural networks. By leveraging the Hadamard convolution theorem, convolutions are performed as element-wise products in the HT domain, with HT computations implementable on quantum hardware via Hadamard gates and a sign-recovery step to preserve information. The HT-perceptron layer uses multiple HT paths, trainable scaling, channel-wise 1x1 convolutions, and trainable soft-thresholding in the HT domain, achieving substantial MACs reductions while maintaining or improving accuracy across MNIST, CIFAR-10, and ImageNet-1K. These findings suggest a practical pathway to more efficient CNNs that can leverage quantum accelerators, with open-source code to facilitate broader adoption and experimentation.

Abstract

In this paper, we propose a novel Hadamard Transform (HT)-based neural network layer for hybrid quantum-classical computing. It implements the regular convolutional layers in the Hadamard transform domain. The idea is based on the HT convolution theorem which states that the dyadic convolution between two vectors is equivalent to the element-wise multiplication of their HT representation. Computing the HT is simply the application of a Hadamard gate to each qubit individually, so the HT computations of our proposed layer can be implemented on a quantum computer. Compared to the regular Conv2D layer, the proposed HT-perceptron layer is computationally more efficient. Compared to a CNN with the same number of trainable parameters and 99.26\% test accuracy, our HT network reaches 99.31\% test accuracy with 57.1\% MACs reduced in the MNIST dataset; and in our ImageNet-1K experiments, our HT-based ResNet-50 exceeds the accuracy of the baseline ResNet-50 by 0.59\% center-crop top-1 accuracy using 11.5\% fewer parameters with 12.6\% fewer MACs.

A Hybrid Quantum-Classical Approach based on the Hadamard Transform for the Convolutional Layer

TL;DR

The paper tackles the computational burden of convolutional layers by introducing a Hadamard Transform (HT)-based HT-perceptron layer for hybrid quantum-classical neural networks. By leveraging the Hadamard convolution theorem, convolutions are performed as element-wise products in the HT domain, with HT computations implementable on quantum hardware via Hadamard gates and a sign-recovery step to preserve information. The HT-perceptron layer uses multiple HT paths, trainable scaling, channel-wise 1x1 convolutions, and trainable soft-thresholding in the HT domain, achieving substantial MACs reductions while maintaining or improving accuracy across MNIST, CIFAR-10, and ImageNet-1K. These findings suggest a practical pathway to more efficient CNNs that can leverage quantum accelerators, with open-source code to facilitate broader adoption and experimentation.

Abstract

In this paper, we propose a novel Hadamard Transform (HT)-based neural network layer for hybrid quantum-classical computing. It implements the regular convolutional layers in the Hadamard transform domain. The idea is based on the HT convolution theorem which states that the dyadic convolution between two vectors is equivalent to the element-wise multiplication of their HT representation. Computing the HT is simply the application of a Hadamard gate to each qubit individually, so the HT computations of our proposed layer can be implemented on a quantum computer. Compared to the regular Conv2D layer, the proposed HT-perceptron layer is computationally more efficient. Compared to a CNN with the same number of trainable parameters and 99.26\% test accuracy, our HT network reaches 99.31\% test accuracy with 57.1\% MACs reduced in the MNIST dataset; and in our ImageNet-1K experiments, our HT-based ResNet-50 exceeds the accuracy of the baseline ResNet-50 by 0.59\% center-crop top-1 accuracy using 11.5\% fewer parameters with 12.6\% fewer MACs.
Paper Structure (10 sections, 2 theorems, 11 equations, 3 figures, 9 tables, 3 algorithms)

This paper contains 10 sections, 2 theorems, 11 equations, 3 figures, 9 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Methodology
  3. Background: Hybrid quantum-classical approach for Hadamard Transform
  4. Hadamard Convolution Theorem
  5. HT-Perceptron Layer
  6. Experimental Results
  7. Conclusion
  8. Proof of Lemma \ref{['lemma: HT positive']}
  9. Proof of Theorem \ref{['Hadamard convolution theorem']} Hadamard convolution theorem
  10. ResNet Structures Used in Experiments

Key Result

Lemma 2.1

Let $\mathbf{x}=[x_0\ x_1\ \ldots\ x_{N-1}]^T$ and $\mathbf{X} = \mathcal{H}(\mathbf{x})=[X_0\ X_1\ \ldots\ X_{N-1}]^T$. If $x_0 > \sum_{k=1}^{N-1}|x_k|$, then $X_k>0$ for $k=0, 1, ..., N-1$.

Figures (3)

  • Figure 1: Structure of the proposed HT-perceptron layer for a tensor in $\mathbb{R}^{C\times W\times H}$. The HT2D and IHT2D are implemented using the quantum computer \ref{['alg: QHT2D']} (or the classical fast approach) and multiplications and soft-thresholding operations of the network are implemented using the classical approach. We have parallel multiple paths to increase the number of trainable parameters. Each path corresponds to a convolutional kernel. If we want to change the number of output channels, we can change the number of kernels at each channel-wise processing.
  • Figure 2: Procedure of each path in the HT-Perceptron layer. Each entry along the width and the height is processed individually in the Hadamard domain, so we don't need to apply the permutation as the Walsh-Hadamard transform.
  • Figure 3: Training on ImageNet-1K. Curves denote the validation center-crop top-1 errors.

Theorems & Definitions (4)

  • Lemma 2.1
  • Theorem 2.2: Hadamard convolution theorem
  • proof
  • proof