RiskLabs: Predicting Financial Risk Using Large Language Model based on Multimodal and Multi-Sources Data

TL;DR

RiskLabs tackles financial risk prediction by integrating multimodal inputs from Earnings Conference Calls, market time-series, and news using an LLM-enabled framework. It introduces four modules—the Earnings Conference Call Encoder, Time-Series Encoder, News-Market-Reactions Encoder, and a Multimodal Fusion block—that feed into a multi-task predictor forecasting volatility over $3$, $7$, $15$, and $30$ days and VaR. Empirical results show RiskLabs outperforms traditional baselines on short- and mid-term horizons and demonstrates that LLMs are most effective as assistants that organize and interpret diverse data rather than as standalone predictors. The work highlights practical implications for timelier, data-rich financial risk assessment and outlines promising directions, including Bayesian-VaR, dynamic training windows, and enhanced news-based signals.

Abstract

The integration of Artificial Intelligence (AI) techniques, particularly large language models (LLMs), in finance has garnered increasing academic attention. Despite progress, existing studies predominantly focus on tasks like financial text summarization, question-answering, and stock movement prediction (binary classification), the application of LLMs to financial risk prediction remains underexplored. Addressing this gap, in this paper, we introduce RiskLabs, a novel framework that leverages LLMs to analyze and predict financial risks. RiskLabs uniquely integrates multimodal financial data, including textual and vocal information from Earnings Conference Calls (ECCs), market-related time series data, and contextual news data to improve financial risk prediction. Empirical results demonstrate RiskLabs' effectiveness in forecasting both market volatility and variance. Through comparative experiments, we examine the contributions of different data sources to financial risk assessment and highlight the crucial role of LLMs in this process. We also discuss the challenges associated with using LLMs for financial risk prediction and explore the potential of combining them with multimodal data for this purpose.

Figures (7)

• Figure 1: This figure illustrates the RiskLabs Framework. The model accepts multiple-source inputs: Earnings Conference Call Audio & Transcript, Daily News, and Time Series Data. The second area visualizes the model's pipeline to encode diverse sources and illustrates how LLMs are applied for data analysis. The third area describes how the model consolidates outputs from both embeddings and LLM analysis for use in subsequent stages. The model will perform multi-task learning: our RiskLabs will predict the Volatility of different terms and VaR in the meantime.
• Figure 2: visualizes the process of fine-grained information extraction from ECC transcript.
• Figure 3: This figure illustrates the pipeline for enriching the news information. First, the pipeline will analyze the sentiments from the target news. Then, based on the binary questions bank designed for different topics, Pipeline can extract the information and answer these questions. Finally, if the pipeline could capture the signal of a specific topic, it would also give feedback on the potential market response.
• Figure 4: This diagram illustrates the process by which the News Analyzer assesses similarities across various news collections. As news items pass through the enrichment pipeline, they are tagged with multiple attributes. Each circle in the figure represents one of these attributes. To identify similar news collections from historical data, the analyzer starts by comparing these attributes and filtering out certain ones. Subsequently, it evaluates the similarity among the remaining attributes to determine the connections between different news collections.
• Figure 5: This figure illustrates the methodology for modeling the impact of earnings conference calls on the stock market when earnings conference calls are not available. It uses a curve to represent the time decay effect of an earnings conference call's influence over time. When an earnings conference call is initially issued, its impact on the market is at its peak. Over time, this influence gradually diminishes.