HIT-ROCKET: Hadamard-vector Inner-product Transformer for ROCKET

TL;DR

HIT-ROCKET introduces a Hadamard-vector inner-product transformer for ROCKET that uses orthogonal Hadamard kernels to extract PPV features with fewer kernels and lower computation. By leveraging $H^T H = N I$ and kernel orthogonality, it achieves improved robustness and efficiency, enabling embedding on ultra-low-power devices while maintaining compatibility with standard classifiers. Empirical results on the UCR time series datasets show at least 5% F1 improvement over ROCKET and around 50% shorter training time than miniROCKET under identical hyperparameters, with strong noise robustness and effective one-class performance. The work provides an open-source implementation and discusses extensions such as non-uniform dilation and differential features, paving the way for efficient edge deployment of time-series classifiers.

Abstract

Time series classification holds broad application value in communications, information countermeasures, finance, and medicine. However, state-of-the-art (SOTA) methods-including HIVE-COTE, Proximity Forest, and TS-CHIEF-exhibit high computational complexity, coupled with lengthy parameter tuning and training cycles. In contrast, lightweight solutions like ROCKET (Random Convolutional Kernel Transform) offer greater efficiency but leave substantial room for improvement in kernel selection and computational overhead. To address these challenges, we propose a feature extraction approach based on Hadamard convolutional transform, utilizing column or row vectors of Hadamard matrices as convolution kernels with extended lengths of varying sizes. This enhancement maintains full compatibility with existing methods (e.g., ROCKET) while leveraging kernel orthogonality to boost computational efficiency, robustness, and adaptability. Comprehensive experiments on multi-domain datasets-focusing on the UCR time series dataset-demonstrate SOTA performance: F1-score improved by at least 5% vs. ROCKET, with 50% shorter training time than miniROCKET (fastest ROCKET variant) under identical hyperparameters, enabling deployment on ultra-low-power embedded devices. All code is available on GitHub.

Figures (4)

• Figure 1: This figure presents the performance ranking results of experiments conducted on 20 datasets from the UCR archive, comparing the proposed method with 5 main competing methods (note: full coverage of HIVE-COTE experiments was not feasible due to computational constraints). (A) shows the accuracy ranking, where the proposed HIT-ROCKET method rankssthird; (B) displays the training and inference time ranking, with HIT-ROCKET ranking first.
• Figure 2: (A) shows the classification performance comparison between HIT-ROCKET and miniROCKET using a ridge regression classifier at the three feature levels (0.5K, 3K, 8K); (B) displays the comparison using a logistic regression classifier at the same feature levels; (C) and (D) present the variance of F1 scores for HIT-ROCKET and miniROCKET under ridge regression and logistic regression, respectively, as SNR varies from 20 dB to 50 dB.
• Figure 3: (A) Average F1 score comparison using ridge regression; (B) Average F1 score comparison using logistic regression; (C) Variance of F1 scores for ridge regression; (D) Variance of F1 scores for logistic regression. HIT-ROCKET consistently shows more stable performance across different SNR levels.
• Figure 4: (A) shows the average training F1 scores (left) and average training times (right) of HIT-ROCKET as functions of feature dimension using a ridge regression classifier; (B) displays the same metrics using a logistic regression classifier.