Foresight Learning for SEC Risk Prediction

TL;DR

The paper tackles probabilistic forecasting of risk materialization from SEC disclosures by turning qualitative risk narratives into quantitative, time-bounded supervision. It introduces a fully automated data generation pipeline and applies Foresight Learning to train a compact LLM (Qwen3-32B) to output calibrated probabilities $P( ext{materialize by } s)$ using only information available at time $t$ and outcomes resolved from future disclosures. Empirically, the fine-tuned model substantially improves over pretrained baselines, a naive base-rate, and frontier models (including GPT-5) across probabilistic metrics such as the Brier score and calibration error, with notable gains in interpretability via reliability curves. The work demonstrates that scalable, domain-specific expert models can be trained directly from unstructured chronological text without proprietary data or manual annotation, offering a general pathway for calibrated decision-relevant signals from enterprise documents, and provides an open evaluation dataset for reproducibility.

Abstract

Risk disclosures in SEC filings describe potential adverse events but rarely quantify their likelihood, limiting their usefulness for probabilistic analysis. A central obstacle is the absence of large-scale, risk-level supervision linking disclosed risks to realized outcomes. We introduce a fully automated data generation pipeline that converts qualitative SEC risk disclosures into temporally grounded supervision using only public data. For each filing, the pipeline generates firm-specific, time-bounded risk queries from the Risk Factors section and labels them by automatically resolving outcomes against subsequent disclosures. Using this dataset of risk queries and outcomes grounded in SEC filings, we train a compact large language model to estimate the probability that a disclosed risk will materialize within a specified horizon. Despite its modest size, the resulting model substantially improves over pretrained and heuristic baselines, and outperforms frontier general-purpose models, including GPT-5, on probabilistic accuracy and calibration. More broadly, this work demonstrates that Foresight Learning enables scalable and fully automated training of domain-specific expert models using only raw, chronological, in-domain text -- without proprietary data, external corpora, or manual annotation. The resulting models achieve frontier-level performance while remaining deployable on a single GPU. This result suggests a general pathway for learning calibrated, decision-relevant signals from naturally occurring enterprise documents. To support transparency and reproducibility, we open-source the evaluation dataset used in this study. Evaluation Data: https://huggingface.co/datasets/LightningRodLabs/sec_risk_questions_test_set Data Generation Platform: https://lightningrod.ai/ SDK: https://github.com/lightning-rod-labs/lightningrod-python-sdk

Figures (2)

• Figure 1: Aggregate performance on the held-out test set
• Figure 2: Reliability diagram on the test set showing empirical materialization rates as a function of predicted risk probabilities.