Graph Generation Powered with LLMs for Boosting Multivariate Time-Series Representation Learning

TL;DR

The paper tackles the problem of biased graph generation for multivariate time-series (MTS) data by introducing K-Link, a framework that leverages a Knowledge-Link graph derived from Large Language Models (LLMs) to inject universal sensor knowledge into graph construction. It pairs a knowledge-link branch, built from sensor- and label-level prompts, with a graph-alignment module that transfers knowledge to MTS graphs through sensor-level and label-level node alignments and edge alignment, optimizing a combined loss L = L_D + λ_S L_S + λ_L L_L + λ_E L_E. Empirical results across regression (e.g., CMAPSS FD002/FD004) and classification tasks (HAR, ISRUC, WISDM, etc.) show that K-Link consistently outperforms state-of-the-art baselines and domain-specific methods, with ablations confirming the importance of knowledge-link components and alignments. The approach demonstrates that incorporating structured, universal knowledge from LLMs can markedly improve graph quality and downstream representation learning, offering better generalization and practical applicability for sensor networks and MTS tasks.

Abstract

Sourced from multiple sensors and organized chronologically, Multivariate Time-Series (MTS) data involves crucial spatial-temporal dependencies. To capture these dependencies, Graph Neural Networks (GNNs) have emerged as powerful tools. As explicit graphs are not inherent to MTS data, graph generation becomes a critical first step in adapting GNNs to this domain. However, existing approaches often rely solely on the data itself for MTS graph generation, leaving them vulnerable to biases from small training datasets. This limitation hampers their ability to construct effective graphs, undermining the accurate modeling of underlying dependencies in MTS data and reducing GNN performance in this field. To address this challenge, we propose a novel framework, K-Link, leveraging the extensive universal knowledge encoded in Large Language Models (LLMs) to reduce biases for powered MTS graph generation. To harness the knowledge within LLMs, such as physical principles, we design and extract a \textit{Knowledge-Link graph} that captures universal knowledge of sensors and their linkage. To empower MTS graph generation with the knowledge-link graph, we further introduce a graph alignment module that transfers universal knowledge from the knowledge-link graph to the graph generated from MTS data. This enhances the MTS graph quality, ensuring effective representation learning for MTS data. Extensive experiments demonstrate the efficacy of K-Link for superior performance on various MTS tasks.

Figures (7)

• Figure 1: Graph generation to capture sensor correlations for MTS data. (A): Graph from MTS data alone is biased by incorrect sensor correlations. Sensor features of fan speed are closer to pressure than temperature, resulting in biased edge connections. (B): LLMs can encode the physical principle that fan speed correlates with temperature, not pressure. This ensures that in the feature space, fan speed should be close with temperature while being separated from pressure. By incorporating this knowledge, sensor feature distributions of MTS data can be enhanced with effective sensor correlations, leading to improved MTS graph with correct edges.
• Figure 2: The overall framework, starting with an MTS graph generation and GNN branch for learning representations. In the knowledge-link branch, a knowledge-link graph is extracted from LLMs to represent universal knowledge. To unlock LLMs' potential, we design sensor-level prompts to extract sensor knowledge and label-level prompts to further enhance the sensor knowledge by considering category information. The knowledge-link graph is then defined with sensor knowledge as nodes and their semantic relationships as edges. To leverage this universal knowledge, a graph alignment module—comprising node and edge alignment—is introduced, facilitating the comprehensive transfer of knowledge from the knowledge-link graph to the MTS-generated graph, thereby enhancing MTS graph generation.
• Figure 3: Prompt description.
• Figure 4: The graph alignment, including node and edge alignment, to comprehensively transfer the universal knowledge from the knowledge-link graph to the MTS graph. Node alignment is further divided into sensor-level and label-level alignment to ensure a balanced and effective transfer of the sensor knowledge at both levels.
• Figure 5: Sensitivity analysis for sensor-level, label-level, and edge alignment in regression tasks (Lower values of both indicators are better).