InstructTime++: Time Series Classification with Multimodal Language Modeling via Implicit Feature Enhancement

TL;DR

This work reframes time series classification as a multimodal generative problem by translating numeric sequences and textual context into a language-model–driven task that generates class-descriptive text. It introduces InstructTime, which discretizes time series with Vector-Quantized tokens, aligns modalities via an MLP-based projection, and pre-trains an LM with cross-domain objectives before task-specific fine-tuning. Building on this, InstructTime++ adds implicit feature modeling through statistical and visual feature toolkits that are translated into natural language and fused within the prompt, addressing the limited inductive bias of LMs for temporal data. Across diverse benchmark datasets, InstructTime++ consistently achieves state-of-the-art results, demonstrating improved robustness, cross-domain generalization, and data efficiency through explicit and implicit multimodal reasoning.

Abstract

Most existing time series classification methods adopt a discriminative paradigm that maps input sequences directly to one-hot encoded class labels. While effective, this paradigm struggles to incorporate contextual features and fails to capture semantic relationships among classes. To address these limitations, we propose InstructTime, a novel framework that reformulates time series classification as a multimodal generative task. Specifically, continuous numerical sequences, contextual textual features, and task instructions are treated as multimodal inputs, while class labels are generated as textual outputs by tuned language models. To bridge the modality gap, InstructTime introduces a time series discretization module that converts continuous sequences into discrete temporal tokens, together with an alignment projection layer and a generative self-supervised pre-training strategy to enhance cross-modal representation alignment. Building upon this framework, we further propose InstructTime++, which extends InstructTime by incorporating implicit feature modeling to compensate for the limited inductive bias of language models. InstructTime++ leverages specialized toolkits to mine informative implicit patterns from raw time series and contextual inputs, including statistical feature extraction and vision-language-based image captioning, and translates them into textual descriptions for seamless integration. Extensive experiments on multiple benchmark datasets demonstrate the superior performance of InstructTime++.

Figures (9)

• Figure 1: Illustrating the overall pipeline of proposed InstructTime.
• Figure 2: Illustration of the overall framework of the proposed InstructTime.
• Figure 3: Illustration of the newly proposed prompt template.
• Figure 4: Illustration of the framework of extended InstructTime++.
• Figure 5: Illustration of the newly proposed prompt template of extended InstructTime++.