POCKET: Pruning Random Convolution Kernels for Time Series Classification from a Feature Selection Perspective

TL;DR

POCKET addresses the practical difficulty of pruning the large pool of random convolution kernels in ROCKET/MINIROCKET without incurring prohibitive evaluation costs. It frames pruning as a group elastic-net problem at the classifier level and introduces an ADMM-based solver, then a two-stage accelerated method, POCKET, that first enforces group sparsity to remove entire kernel groups and then (optionally) refits with an $l_2$-regularized classifier on the remaining features. Empirical results on the UCR time series benchmarks show POCKET can remove over 60% of kernels with minimal or even improved accuracy, while delivering an approx. 11x speedup over the ADMM baseline and outperforming S-ROCKET in most settings. The approach provides a controllable, resource-aware pruning mechanism for ROCKET-family models, with practical implications for deploying fast, accurate TSC on constrained devices.

Abstract

In recent years, two competitive time series classification models, namely, ROCKET and MINIROCKET, have garnered considerable attention due to their low training cost and high accuracy. However, they rely on a large number of random 1-D convolutional kernels to comprehensively capture features, which is incompatible with resource-constrained devices. Despite the development of heuristic algorithms designed to recognize and prune redundant kernels, the inherent time-consuming nature of evolutionary algorithms hinders efficient evaluation. To efficiently prune models, this paper eliminates feature groups contributing minimally to the classifier, thereby discarding the associated random kernels without direct evaluation. To this end, we incorporate both group-level ($l_{2,1}$-norm) and element-level ($l_2$-norm) regularizations to the classifier, formulating the pruning challenge as a group elastic net classification problem. An ADMM-based algorithm is initially introduced to solve the problem, but it is computationally intensive. Building on the ADMM-based algorithm, we then propose our core algorithm, POCKET, which significantly speeds up the process by dividing the task into two sequential stages. In Stage 1, POCKET utilizes dynamically varying penalties to efficiently achieve group sparsity within the classifier, removing features associated with zero weights and their corresponding kernels. In Stage 2, the remaining kernels and features are used to refit a $l_2$-regularized classifier for enhanced performance. Experimental results on diverse time series datasets show that POCKET prunes up to 60% of kernels without a significant reduction in accuracy and performs 11$\times$ faster than its counterparts. Our code is publicly available at https://github.com/ShaowuChen/POCKET.

Figures (7)

• Figure 1: Overall diagram of POCKET. TOP: Stage 1, remove feature groups that minimally contribute to the $\ell_{2,1}$-regularized classifier, thereby eliminating the corresponding kernels. BOTTOM: (optional) Stage 2, refit a $\ell_2$-regularized classifier using the remaining (kernels and) features for enhanced performance.
• Figure 2: Models pruned by POCKET vs. Reduced-kernel models trained from scratch.
• Figure 3: Average $\ell_2$ regularization strength of models across datasets involved in Figure \ref{['VSFromScratch1']}.
• Figure 4: Pruning ROCKET-PPV-MAX for the first four datasets under different pruning rates.
• Figure 5: Accuracy comparison across 128 UCR datasets when 90% of kernels of ROCKET-PPV are pruned by POCKET.