MemPromptTSS: Persistent Prompt Memory for Iterative Multi-Granularity Time Series State Segmentation

TL;DR

MemPromptTSS addresses the challenge of sparse, multi-granularity time-series segmentation by introducing a persistent prompt memory stored in a memory bank. A memory encoder converts prompts and their local context into memory tokens that persist across iterations, enabling global consistency as predictions condition on all accumulated prompts. The framework fuses time-series embeddings with memory tokens via a Two-Way Transformer decoder, supporting label and boundary prompts and iterative refinement across subsequences. Empirical results on six wearable and industrial datasets show substantial accuracy gains over baselines in single- and multi-granularity settings, and stronger refinement in iterative inference, highlighting the practical value of persistent memory for prompt-guided segmentation.

Abstract

Web platforms, mobile applications, and connected sensing systems generate multivariate time series with states at multiple levels of granularity, from coarse regimes to fine-grained events. Effective segmentation in these settings requires integrating across granularities while supporting iterative refinement through sparse prompt signals, which provide a compact mechanism for injecting domain knowledge. Yet existing prompting approaches for time series segmentation operate only within local contexts, so the effect of a prompt quickly fades and cannot guide predictions across the entire sequence. To overcome this limitation, we propose MemPromptTSS, a framework for iterative multi-granularity segmentation that introduces persistent prompt memory. A memory encoder transforms prompts and their surrounding subsequences into memory tokens stored in a bank. This persistent memory enables each new prediction to condition not only on local cues but also on all prompts accumulated across iterations, ensuring their influence persists across the entire sequence. Experiments on six datasets covering wearable sensing and industrial monitoring show that MemPromptTSS achieves 23% and 85% accuracy improvements over the best baseline in single- and multi-granularity segmentation under single iteration inference, and provides stronger refinement in iterative inference with average per-iteration gains of 2.66 percentage points compared to 1.19 for PromptTSS. These results highlight the importance of persistent memory for prompt-guided segmentation, establishing MemPromptTSS as a practical and effective framework for real-world applications.

Figures (6)

• Figure 1: Problem setup and iterative refinement in MemPromptTSS. On the left, segmentation is performed over subsequences rather than a single sliding window, with prompts provided as label or boundary cues. On the right, iterative refinement illustrates how additional prompts guide the model to update predictions across multiple granularities, from coarse categories (e.g., move, sit) to fine actions (e.g., walk, run, jump).
• Figure 2: Overview of the MemPromptTSS framework. At each iteration, prompts are encoded and written into a per-subsequence memory bank (Memory Write). For every window, the state decoder integrates time series embeddings with memory tokens to produce predictions (Memory Read). Segmentation quality improves progressively as more prompts are provided across iterations.
• Figure 3: Comparison of training and inference times per batch on the PAMAP2 dataset.
• Figure 4: Segmentation performance under the multiple-iteration inference setting, evaluated on two datasets (PAMAP2 and Pump V35), each with both single and multiple granularities of states. Top row: Test accuracy (ACC, %). Bottom row: Iteration $\Delta$ ACC (percentage points, pp).
• Figure 5: Ablation study on the impact of context window length $T_{ctx}$ on segmentation accuracy for PAMAP2 and PumpV35.