ConvTimeNet: A Deep Hierarchical Fully Convolutional Model for Multivariate Time Series Analysis

TL;DR

ConvTimeNet tackles core time series analysis challenges by unifying adaptive local pattern extraction with multi-scale global dependency modeling in a deep, pure convolutional architecture. It introduces deformable patch embedding to capture informative local patterns and a stack of large-kernel fully convolutional blocks with a learnable residual mechanism to expand the receptive field without attention. The approach delivers competitive or superior performance on both time series classification and forecasting across multiple datasets, often surpassing Transformer-based baselines while maintaining efficiency. The results highlight that convolutional architectures remain strong contenders for time series tasks when combined with adaptive patching and hierarchical multi-scale design.

Abstract

Designing effective models for learning time series representations is foundational for time series analysis. Many previous works have explored time series representation modeling approaches and have made progress in this area. Despite their effectiveness, they lack adaptive perception of local patterns in temporally dependent basic units and fail to capture the multi-scale dependency among these units. Instead of relying on prevalent methods centered around self-attention mechanisms, we propose ConvTimeNet, a hierarchical pure convolutional model designed for time series analysis. ConvTimeNet introduces a deformable patch layer that adaptively perceives local patterns of temporally dependent basic units in a data-driven manner. Based on the extracted local patterns, hierarchical pure convolutional blocks are designed to capture dependency relationships among the representations of basic units at different scales. Moreover, a large kernel mechanism is employed to ensure that convolutional blocks can be deeply stacked, thereby achieving a larger receptive field. In this way, local patterns and their multi-scale dependencies can be effectively modeled within a single model. Extensive experiments comparing a wide range of different types of models demonstrate that pure convolutional models still exhibit strong viability, effectively addressing the aforementioned two challenges and showing superior performance across multiple tasks. The code is available for reproducibility.

Figures (5)

• Figure 1: Illustration of the newly proposed ConvTimeNet.
• Figure 2: Critical difference diagram over the mean ranks of ConvTimeNet, baseline methods of classification task.
• Figure 3: Model performance comparison in time series forecasting task.
• Figure 4: Performance comparison between our full-convolution block and Transformer encoder block.
• Figure 5: The visualization of the forecasting case, generated by various models under the input-336-predict-336 setting, is presented. The black lines represent the ground truth, while the orange lines represent the predicted values.