SEDformer: Event-Synchronous Spiking Transformers for Irregular Telemetry Time Series Forecasting

TL;DR

SEDformer is presented, an SED-enhanced Spiking Transformer for telemetry IMTS forecasting that couples a SED-based Spike Encoder, an Event-Preserving Temporal Downsampling module, and a stack of SED-based Spike Transformer blocks enable intra-series dependency modeling with a membrane-based linear attention driven by EA-LIF spiking features.

Abstract

Telemetry streams from large-scale Internet-connected systems (e.g., IoT deployments and online platforms) naturally form an irregular multivariate time series (IMTS) whose accurate forecasting is operationally vital. A closer examination reveals a defining Sparsity-Event Duality (SED) property of IMTS, i.e., long stretches with sparse or no observations are punctuated by short, dense bursts where most semantic events (observations) occur. However, existing Graph- and Transformer-based forecasters ignore SED: pre-alignment to uniform grids with heavy padding violates sparsity by inflating sequences and forcing computation at non-informative steps, while relational recasting weakens event semantics by disrupting local temporal continuity. These limitations motivate a more faithful and natural modeling paradigm for IMTS that aligns with its SED property. We find that Spiking Neural Networks meet this requirement, as they communicate via sparse binary spikes and update in an event-driven manner, aligning naturally with the SED nature of IMTS. Therefore, we present SEDformer, an SED-enhanced Spiking Transformer for telemetry IMTS forecasting that couples: (1) a SED-based Spike Encoder converts raw observations into event synchronous spikes using an Event-Aligned LIF neuron, (2) an Event-Preserving Temporal Downsampling module compresses long gaps while retaining salient firings and (3) a stack of SED-based Spike Transformer blocks enable intra-series dependency modeling with a membrane-based linear attention driven by EA-LIF spiking features. Experiments on public telemetry IMTS datasets show that SEDformer attains state-of-the-art forecasting accuracy while reducing energy and memory usage, providing a natural and efficient path for modeling IMTS.

Figures (7)

• Figure 1: Five-month daily Wikipedia page views for two articles, exhibiting both sparsity and dense events. For brevity, we abbreviate the titles as "Gustavo" and "Datei". Beyond article page views, telemetry IMTS also arises in IoT sensing, messaging, video streaming, and clickstream, where sparsity--event duality similarly holds.
• Figure 2: The architecture of SEDformer. SED-based Spike Encoder (SED-SE) initially converts IMTS into event-synchronous spike trains via an Event-Aligned LIF (EA-LIF) neuron. Event-Preserving Temporal Downsampling (EPTD) module then retains salient spikes while collapsing long gaps. A stack of SED-based Spike Transformer (SED-ST) blocks subsequently learns temporal dependencies via membrane-based SED-Attention mechanism, and Masked Time Aggregation (MTA) summarizes each variate over observed events. A lightweight Decoder finally maps the summaries to forecasts at future times.
• Figure 3: The structure of the (a) vanilla LIF and the (b) Event-Aligned LIF (EA-LIF) we propose.
• Figure 4: Efficiency analysis on the Wiki2000 dataset with Sparsifying Rate = $\mathbf{25\%}$. “OPs” refers to SOPs in SNN and FLOPs in ANN. “SOPs” is the synaptic operations of SEDformer. “FLOPs” denotes the floating point operations of other ANN baselines.
• Figure 5: Spike trains generated by Delta Spike Encoder and Convolutional Spike Encoder with vanilla LIF neuron, as well as the proposed SED-based Spike Encoder with EA-LIF.