Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection

Xinlei Wang, Maike Feng, Jing Qiu, Jinjin Gu, Junhua Zhao

TL;DR

This paper utilizes LLM-based agents to iteratively filter out irrelevant news and employ human-like reasoning to evaluate predictions, which enables the model to analyze complex events, such as unexpected incidents and shifts in social behavior.

Abstract

This paper introduces a novel approach that leverages Large Language Models (LLMs) and Generative Agents to enhance time series forecasting by reasoning across both text and time series data. With language as a medium, our method adaptively integrates social events into forecasting models, aligning news content with time series fluctuations to provide richer insights. Specifically, we utilize LLM-based agents to iteratively filter out irrelevant news and employ human-like reasoning to evaluate predictions. This enables the model to analyze complex events, such as unexpected incidents and shifts in social behavior, and continuously refine the selection logic of news and the robustness of the agent's output. By integrating selected news events with time series data, we fine-tune a pre-trained LLM to predict sequences of digits in time series. The results demonstrate significant improvements in forecasting accuracy, suggesting a potential paradigm shift in time series forecasting through the effective utilization of unstructured news data.

From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection

TL;DR

This paper utilizes LLM-based agents to iteratively filter out irrelevant news and employ human-like reasoning to evaluate predictions, which enables the model to analyze complex events, such as unexpected incidents and shifts in social behavior.

Abstract

This paper introduces a novel approach that leverages Large Language Models (LLMs) and Generative Agents to enhance time series forecasting by reasoning across both text and time series data. With language as a medium, our method adaptively integrates social events into forecasting models, aligning news content with time series fluctuations to provide richer insights. Specifically, we utilize LLM-based agents to iteratively filter out irrelevant news and employ human-like reasoning to evaluate predictions. This enables the model to analyze complex events, such as unexpected incidents and shifts in social behavior, and continuously refine the selection logic of news and the robustness of the agent's output. By integrating selected news events with time series data, we fine-tune a pre-trained LLM to predict sequences of digits in time series. The results demonstrate significant improvements in forecasting accuracy, suggesting a potential paradigm shift in time series forecasting through the effective utilization of unstructured news data.
Paper Structure (23 sections, 1 equation, 16 figures, 9 tables)

This paper contains 23 sections, 1 equation, 16 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Rethinking Time Series Forecasting Problem and Elements.
  5. Analytical Agent for Aggregation and Reasoning of Contextual News Information.
  6. Overall Pipeline
  7. Experiments
  8. Data preparation
  9. Results
  10. Conclusion and Discussion
  11. Appendix / Supplemental Material
  12. Experimental Setting / Details
  13. Example of Numerical Input for Fine-tuning LLM
  14. Example of Textual Input for Fine-tuning LLM
  15. News Sources and Details
  16. ...and 8 more sections

Figures (16)

  • Figure 1: Integrating textual information in time series forecasting. (A) We retrieve relevant original news and supplementary information from our comprehensive database based on information such as the geographic location and time frame of the prediction task. (B) LLM-based agents analyze and select relevant news for different forecasting horizons. (C & D) The selected news and contextual information are combined with time series data for fine-tuning the LLM forecasting model. (E) Discrepancies between predictions and ground truth trigger a review of historical news and data to reprocess missed information and refine reasoning logic.
  • Figure 2: Relationship between news and time series. This figure illustrates the news filtered by the reasoning agents, using the example of Australia's state-level electricity demand. It features load data in Victoria state and selected news from June 10 to 12, 2019. The black arrow indicates time-specific events, the blue curve shows load fluctuations. The x-axis represents time, and the y-axis displays load values in kilowatts. The blue box displays the short-term impact news and long-term impact news selected by the reasoning agent (e.g., traffic incidents or new construction projects).
  • Figure 3: Example of prompt designs for each iteration during fine-tuning. Step 1 involves the reasoning agent selecting news using default logic. Step 2 evaluates predictions based on validation sets to refine the logic. In step 3, the updated logic directs data pairing for the next iteration. Full prompts are shown in Appendix \ref{['sec:prompt details']}.
  • Figure 4: Overall pipeline iteratively combines news reasoning agents, fine-tuning, and evaluation agents.
  • Figure 5: Day-ahead Australia electricity demand forecasting with/without news. The horizontal axis is the time index (half hour). Actual load demands are in solid blue, predictions with news in solid red, and predictions without news in dashed green. (a) Sydney's lockdown news effects; (b) Residential electricity consumption behavior news effects; (c) Anticipated power outage news effects.
  • ...and 11 more figures