Beyond Fixed Patches: Enhancing GPTs for Financial Prediction with Adaptive Segmentation and Learnable Wavelets

TL;DR

This study proposes a novel framework that enhances pretrained transformer capabilities for temporal sequence modeling through dynamic patch segmentation and learnable wavelet transform modules and introduces adaptive patch segmentation that partitions temporal sequences while preserving pattern integrity.

Abstract

The extensive adoption of web technologies in the finance and investment sectors has led to an explosion of financial data, which contributes to the complexity of the forecasting task. Traditional machine learning models exhibit limitations in this forecasting task constrained by their restricted model capacity. Recent advances in Generative Pre-trained Transformers (GPTs), with their greatly expanded parameter spaces, demonstrate promising potential for modeling complex dependencies in temporal sequences. However, existing pretraining-based approaches typically focus on fixed-length patch analysis, ignoring market data's multi-scale pattern characteristics. In this study, we propose $\mathbf{GPT4FTS}$, a novel framework that enhances pretrained transformer capabilities for temporal sequence modeling through dynamic patch segmentation and learnable wavelet transform modules. Specifically, we first employ K-means++ clustering based on DTW distance to identify scale-invariant patterns in market data. Building upon pattern recognition results, we introduce adaptive patch segmentation that partitions temporal sequences while preserving pattern integrity. To accommodate time-varying frequency characteristics, we devise a dynamic wavelet transform module that emulates discrete wavelet transformation with enhanced flexibility in capturing time-frequency features. Extensive experiments on real-world financial datasets substantiate the framework's efficacy. The source code is available: \href{https://anonymous.4open.science/r/GPT4FTS-6BCC/}

Figures (5)

• Figure 1: Figures a and d reveal discrepancies in period patterns between financial and electricity data. Figures b and e illustrate deviations in their frequency domain distributions. Figures c and f abstract the differences.
• Figure 2: The architecture of the proposed GPT4FTS model. Part (a) outlines the off-line learning time series scale invariant pattern recognition module, which uses k-means++ clustering and DTW algorithms to match patterns of sequences of different lengths to obtain sequence cuts that minimize the total distance. Part (b) describes the training process using dynamic stepping. Part (c) introduces a learnable wavelet transform module that adaptively decomposes the input sequences.
• Figure 3: The SWT tokenization method employs input padding and incorporates learnable filters with zero-insertion operations.
• Figure 4: The accumulated returns gained in the NASDAQ 100 dataset (2024) by GPT4FTS and selected baselines.
• Figure 5: Performance comparison between fixed and learnable wavelet transforms.