OpenEP: Open-Ended Future Event Prediction

TL;DR

OpenEP reframes future event prediction from fixed-class classification to open-ended generation, enabling diverse questions and unconstrained outcomes. It introduces OpenEPBench, a bilingual dataset with daily hot-topic questions and free-form ground-truth outcomes, and StkFEP, a stakeholder-enhanced retrieval–integration–prediction framework that leverages both relevant and similar events. The approach combines explicit stakeholder extraction for question expansion, long-context information retrieval, and cluster-based integration to manage redundancy, evaluated with novel LLM-based semantic metrics and human validation. Results show current large language models struggle with open-ended FEP, highlighting the need for improved data collection, retrieval strategies, and evaluation protocols to advance practical open-ended forecasting.

Abstract

Future event prediction (FEP) is a long-standing and crucial task in the world, as understanding the evolution of events enables early risk identification, informed decision-making, and strategic planning. Existing work typically treats event prediction as classification tasks and confines the outcomes of future events to a fixed scope, such as yes/no questions, candidate set, and taxonomy, which is difficult to include all possible outcomes of future events. In this paper, we introduce OpenEP (an Open-Ended Future Event Prediction task), which generates flexible and diverse predictions aligned with real-world scenarios. This is mainly reflected in two aspects: firstly, the predictive questions are diverse, covering different stages of event development and perspectives; secondly, the outcomes are flexible, without constraints on scope or format. To facilitate the study of this task, we construct OpenEPBench, an open-ended future event prediction dataset. For question construction, we pose questions from seven perspectives, including location, time, event development, event outcome, event impact, event response, and other, to facilitate an in-depth analysis and understanding of the comprehensive evolution of events. For outcome construction, we collect free-form text containing the outcomes as ground truth to provide semantically complete and detail-enriched outcomes. Furthermore, we propose StkFEP, a stakeholder-enhanced future event prediction framework, that incorporates event characteristics for open-ended settings. Our method extracts stakeholders involved in events to extend questions to gather diverse information. We also collect historically events that are relevant and similar to the question to reveal potential evolutionary patterns. Experiment results indicate that accurately predicting future events in open-ended settings is challenging for existing LLMs.

Figures (14)

• Figure 1: Example of Future Event Prediction.
• Figure 2: Example from the OpenEPBench dataset.
• Figure 3: Data distribution across different types of questions.
• Figure 4: The framework of StkFEP.
• Figure 5: