Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Future-as-Label: Scalable Supervision from Real-World Outcomes

Benjamin Turtel, Paul Wilczewski, Danny Franklin, Kris Skothiem

TL;DR

This work extends reinforcement learning with verifiable rewards to real-world, temporally resolved prediction tasks by introducing Foresight Learning, where predictions at time t rely only on information available up to t and are evaluated after external outcome resolution. A predictor-resolver setup plus a causal information mask enables outcome-based supervision via a proper scoring rule, optimized with Group Relative Policy Optimization (GRPO) over multiple trajectories. Empirically, fine-tuning Qwen3-32B with this framework yields substantial calibration and accuracy gains on synthetic future-event tasks and the Metaculus benchmark, outperforming much larger pretrained models and prompting-based baselines. The approach offers scalable, annotation-free supervision for open-world forecasting, with strong implications for probabilistic forecasting under uncertainty and future directions toward online deployment and richer outcomes.

Abstract

Many real-world prediction problems lack labels observable at prediction time, creating a temporal gap between prediction and outcome that yields supervision only after events resolve. To address this setting, we extend reinforcement learning with verifiable rewards to temporally resolved real-world prediction, and use it to train language models to make probabilistic forecasts under causally masked information with retrospective evaluation using proper scoring rules. Supervision is derived solely from post-resolution outcomes, preserving delayed-reward semantics. On real-world forecasting benchmarks, Qwen3-32B trained using Foresight Learning improves Brier score by 27% and halves calibration error relative to its pretrained baseline, and outperforms Qwen3-235B on both constructed future-event prediction tasks and the Metaculus benchmark despite a 7x parameter disadvantage.

Future-as-Label: Scalable Supervision from Real-World Outcomes

TL;DR

This work extends reinforcement learning with verifiable rewards to real-world, temporally resolved prediction tasks by introducing Foresight Learning, where predictions at time t rely only on information available up to t and are evaluated after external outcome resolution. A predictor-resolver setup plus a causal information mask enables outcome-based supervision via a proper scoring rule, optimized with Group Relative Policy Optimization (GRPO) over multiple trajectories. Empirically, fine-tuning Qwen3-32B with this framework yields substantial calibration and accuracy gains on synthetic future-event tasks and the Metaculus benchmark, outperforming much larger pretrained models and prompting-based baselines. The approach offers scalable, annotation-free supervision for open-world forecasting, with strong implications for probabilistic forecasting under uncertainty and future directions toward online deployment and richer outcomes.

Abstract

Many real-world prediction problems lack labels observable at prediction time, creating a temporal gap between prediction and outcome that yields supervision only after events resolve. To address this setting, we extend reinforcement learning with verifiable rewards to temporally resolved real-world prediction, and use it to train language models to make probabilistic forecasts under causally masked information with retrospective evaluation using proper scoring rules. Supervision is derived solely from post-resolution outcomes, preserving delayed-reward semantics. On real-world forecasting benchmarks, Qwen3-32B trained using Foresight Learning improves Brier score by 27% and halves calibration error relative to its pretrained baseline, and outperforms Qwen3-235B on both constructed future-event prediction tasks and the Metaculus benchmark despite a 7x parameter disadvantage.
Paper Structure (18 sections, 2 equations, 3 figures, 1 table)

This paper contains 18 sections, 2 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reinforcement learning with verifiable rewards
  4. LLM-based forecasting and static supervision
  5. Model-based judges and endogenous rewards
  6. Method
  7. Learning formulation
  8. Objective and optimization
  9. Training protocol
  10. Experimental Setup
  11. Dataset construction
  12. Dataset Statistics and Splits
  13. Task characteristics
  14. Models and training
  15. Baselines
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Overview of Foresight Learning
  • Figure 2: Model calibration and accuracy metrics versus training steps on Metaculus (top) and synthetic future-events (bottom). Shaded regions show 95% bootstrap confidence intervals. Metrics are log score (↑), Brier score (↓), and expected calibration error (ECE; ↓). Performance improves monotonically with training.
  • Figure 3: Brier scores (↓) for different models on Metaculus and synthetic future-events benchmarks. Foresight Learning consistently outperforms both the base and larger pretrained baselines.