VIFO: Visual Feature Empowered Multivariate Time Series Forecasting with Cross-Modal Fusion

TL;DR

VIFO tackles the challenge of cross-variable dependencies in multivariate time series by rendering data as variable-sized images and employing a frozen vision backbone to extract cross-channel patterns. The approach aligns visual features with time-series representations through cross-modal spatiotemporal attention, enabling efficient learning with only about 7.45% of parameters being trainable. Across seven benchmarks, VIFO delivers competitive forecasting accuracy, and ablations show that both visual and time-series modalities contribute to performance, especially for long-horizon tasks. Thiswork demonstrates the viability of integrating pre-trained vision models into time-series forecasting to capture rich spatiotemporal dependencies and cross-modal information.

Abstract

Large time series foundation models often adopt channel-independent architectures to handle varying data dimensions, but this design ignores crucial cross-channel dependencies. Concurrently, existing multimodal approaches have not fully exploited the power of large vision models (LVMs) to interpret spatiotemporal data. Additionally, there remains significant unexplored potential in leveraging the advantages of information extraction from different modalities to enhance time series forecasting performance. To address these gaps, we propose the VIFO, a cross-modal forecasting model. VIFO uniquely renders multivariate time series into image, enabling pre-trained LVM to extract complex cross-channel patterns that are invisible to channel-independent models. These visual features are then aligned and fused with representations from the time series modality. By freezing the LVM and training only 7.45% of its parameters, VIFO achieves competitive performance on multiple benchmarks, offering an efficient and effective solution for capturing cross-variable relationships in

Figures (2)

• Figure 1: Clear Patterns in Electricity and Traffic Time Series Visualization: Periodicity, Lead-Lag Relationships, Anomalous Events.
• Figure 2: The overall structure of VIFO, which simultaneously processes information across temporal and spatial dimensions from both image and time series modalities.