PATRA: Pattern-Aware Alignment and Balanced Reasoning for Time Series Question Answering

TL;DR

The Pattern-Aware Alignment and Balanced Reasoning model (PATRA) is proposed, introducing a pattern-aware mechanism that extracts trend and seasonality patterns from time series to achieve deep alignment.

Abstract

Time series reasoning demands both the perception of complex dynamics and logical depth. However, existing LLM-based approaches exhibit two limitations: they often treat time series merely as text or images, failing to capture the patterns like trends and seasonalities needed to answer specific questions; and when trained on a mix of simple and complex tasks, simpler objectives often dominate the learning process, hindering the development of deep reasoning capabilities. To address these limitations, we propose the Pattern-Aware Alignment and Balanced Reasoning model (PATRA), introducing a pattern-aware mechanism that extracts trend and seasonality patterns from time series to achieve deep alignment. Furthermore, we design a task-aware balanced reward to harmonize learning across tasks of varying difficulty, incentivizing the generation of coherent Chains of Thought. Extensive experiments show that PATRA outperforms strong baselines across diverse Time Series Question Answering (TSQA) tasks, demonstrating superior cross-modal understanding and reasoning capability.

Figures (7)

• Figure 1: Comparison of alignment paradigms in Time Series Question Answering. (a) Single-Modality Reasoning treats time series as text sequences, converting continuous numerical points into text tokens. (b) Multimodal Shallow Alignment introduces time series as a separate modality but relies on simple concatenation of embeddings. (c) Multimodal Deep Alignment addresses these limitations by explicitly decomposing time series into distinct patterns and fusing them with text embeddings via a dedicated module, ensuring that semantic reasoning is firmly grounded in physical data behaviors.
• Figure 2: The framework of PATRA, which contains a Text Encoder for obtaining text embedding from pre-trained representations, a TS Encoder for embedding the time series, a Pattern-Aware Alignment to align the gap between text embedding and time series embedding, and an LLM Backbone to reason the questions.
• Figure 3: Overview of PATRA's two training stages. Alignment Stage performs supervised fine-tuning (SFT) with cross-entropy (CE) loss, while Reasoning-Enhanced Stage applies reinforcement learning, including format and task reward under the GRPO strategy to optimize the policy model.
• Figure 4: Case Study of TS Reasoning Task, comparing the answers of a single-modality LLM (Qwen2.5), a multimodal LLM (ChatTS), and our proposed model (PATRA).
• Figure 5: The Loss of Alignment stage and the Total Reward Curve in Reasoning-Enhanced Stage.