Multi-Source Knowledge-Based Hybrid Neural Framework for Time Series Representation Learning

TL;DR

MKH-Net introduces a multi-source knowledge-based hybrid framework for time series representation learning that combines domain-specific explicit graph knowledge with implicit hypergraph structures. It jointly models spatio-temporal dynamics via three spatial channels—implicit hypergraph, explicit subgraph, and dual-hypergraph—and fuses them with a mixture-of-experts temporal predictor, enabling accurate multi-horizon forecasts and calibrated uncertainty. Empirical results across seven traffic datasets show substantial improvements over state-of-the-art baselines, with notable RMSE reductions and robust uncertainty estimates in the w/Unc-MKH-Net variant. The approach advances practical MTS forecasting by capturing higher-order relationships, subgraph patterns, and time-varying uncertainty, while maintaining scalability to large sensor networks.

Abstract

Accurately predicting the behavior of complex dynamical systems, characterized by high-dimensional multivariate time series(MTS) in interconnected sensor networks, is crucial for informed decision-making in various applications to minimize risk. While graph forecasting networks(GFNs) are ideal for forecasting MTS data that exhibit spatio-temporal dependencies, prior works rely solely on the domain-specific knowledge of time-series variables inter-relationships to model the nonlinear dynamics, neglecting inherent relational structural dependencies among the variables within the MTS data. In contrast, contemporary works infer relational structures from MTS data but neglect domain-specific knowledge. The proposed hybrid architecture addresses these limitations by combining both domain-specific knowledge and implicit knowledge of the relational structure underlying the MTS data using Knowledge-Based Compositional Generalization. The hybrid architecture shows promising results on multiple benchmark datasets, outperforming state-of-the-art forecasting methods. Additionally, the architecture models the time varying uncertainty of multi-horizon forecasts.

Figures (9)

• Figure 1: Overview of MKH-Net framework.
• Figure 2: The hypergraph inference(HgI) module learns complex hierarchical structural-dynamic dependencies among multiple time series variables in a sparse discrete hypergraph structure using similarity metric learning. In this structure, hypernodes represent variables, and hyperedges represent higher-order relations among an arbitrary number of hypernodes. The HgI module is fully differentiable, allowing for efficient optimization using gradient-based algorithms. Simultaneously, the hypergraph representation learning(HgRL) module encodes the structural spatio-temporal inductive biases for modeling the nonlinear dynamics of interconnected sensor networks. In short, the HgRL module learns a latent representations of the continuous-time spatio-temporal hypergraph structures to capture the underlying patterns and trends in the MTS data. For illustration purpose, in the above figure filled circles denote the hypernodes and filled eclipses($e$) denotes the hyperedges.
• Figure 3: The Subgraph representation learning(SgRL) method comprises of two modules: the patch extraction and the subgraph encoder. The patch extraction module partitions spatio-temporal graphs into mutually exclusive k-sets and expands them to include their p-hop neighborhood, creating k-overlapping subgraph patches. The subgraph encoder module processes these subgraph patches to compute node-level representations, which are then mean-pooled across all subgraph patches to obtain the final node representations. For illustration purpose, in the figure above, an arbitrary graph is partitioned into k(3)-mutually exclusive patches. Each patch is then expanded to include its p(1)-hop neighborhood, resulting in k(3)-overlapping subgraph patches.
• Figure 4: The Dual Hypergraph Transformation(DHT) is an effective technique for transforming and analyzing complex spatio-temporal graph structures. It involves the graph-to-hypergraph transformation, which alters the roles of nodes and edges while preserving the shared connectivity pattern and original graph information. This powerful approach is illustrated in the figure above, where numbers(letters) represent the edges(nodes) in the original predefined graph, and hypernodes(hyperedges) in the dual hypergraph.
• Figure 5: Usage of multiple spatial feature extractors has significant benefits by providing a comprehensive and diverse representation of MTS data and reducing the risk of overfitting to a specific aspect of the data by incentivizing each extractor to specialize in a different aspect. This collaborative mechanism design optimizes forecasting error by leveraging the strengths of multiple extractors and fusing their representations to enhance the accuracy and robustness of framework predictions, resulting in more reliable outcomes. In the above figure, subfigures (a), (b), and (c) illustrate the "dual-hypergraph", "explicit subgraph", and "implicit hypergraph" representation learning methods, respectively.