General Transform: A Unified Framework for Adaptive Transform to Enhance Representations

TL;DR

General Transform (GT) introduces a trainable, data-driven framework that adaptively blends multiple discrete transforms to optimize feature representations for a given task. By learning blending weights $p_i$ alongside network parameters, GT can recover standard transforms at specific settings while discovering dataset-specific combinations, achieving improvements in both computer vision and natural language processing with only a small parameter overhead. Empirical results show GT surpasses fixed-transform baselines (e.g., DCTNet, FNet) on ImageNet, SST-2, and CoLA, and can be extended to quantum architectures as Quantum General Transform (QGT) via Linear Combination of Unitaries (LCU). The work demonstrates GT’s robustness, generality across modalities, and potential for quantum-accelerated transform learning, suggesting a versatile and scalable approach for representation learning.

Abstract

Discrete transforms, such as the discrete Fourier transform, are widely used in machine learning to improve model performance by extracting meaningful features. However, with numerous transforms available, selecting an appropriate one often depends on understanding the dataset's properties, making the approach less effective when such knowledge is unavailable. In this work, we propose General Transform (GT), an adaptive transform-based representation designed for machine learning applications. Unlike conventional transforms, GT learns data-driven mapping tailored to the dataset and task of interest. Here, we demonstrate that models incorporating GT outperform conventional transform-based approaches across computer vision and natural language processing tasks, highlighting its effectiveness in diverse learning scenarios.

Figures (5)

• Figure 1: Image classification using RGB images as input, with DCT Xu2020 (a) and GT (b) for feature extraction. Text classification with DFT leethorp2022fnetmixingtokensfourier (c) and GT (d) for token mixing.
• Figure 2: Loss and accuracy curves for image classification on the ImageNet 2012 dataset using DCTNet and GTNet, with 24 (a, b), 48 (c, d), and 64 (e, f) input channels.
• Figure 3: Validation loss ($L$) curve and top-1 accuracy ($A_{\text{Top-1}}$) curve for CoLA (a and b) and SST-2 (c and d) datasets.
• Figure 4: Different unitaries used for QGT: (a) Quantum Fourier Transform (QFT) where $R_k = \left(100e^{i 2\pi / 2^k}\right)$, (b) Clifford+T, (c) Instantaneous Quantum Polynomial (IQP) and (d) Quantum Neural Network (QNN).
• Figure 5: Training and validation loss ($L$) curves (a) and top-1 accuracy ($A_{\text{Top-1}}$) curves (b) for image classification on the ImageNet 2012 dataset using quantum general transform.