Aspen Open Jets: Unlocking LHC Data for Foundation Models in Particle Physics

TL;DR

This paper tackles how to leverage open LHC data to train foundation models for jet physics. It introduces AOJ, a large-scale dataset of 178M CMS jets, and demonstrates pre-training of a jet-focused OmniJet-$\alpha$ model on AOJ, followed by fine-tuning to generate Jets in JetClass under domain shifts. The results show meaningful improvements in downstream generative tasks at small labeled-data budgets and reveal a robust power-law scaling $D^{-\gamma}$ with $0.2<\gamma<0.45$ for many observables. By releasing both the dataset and the jet-based foundation model, the work highlights the practical impact of public collider data for advancing foundation-model approaches in high-energy physics.

Abstract

Foundation models are deep learning models pre-trained on large amounts of data which are capable of generalizing to multiple datasets and/or downstream tasks. This work demonstrates how data collected by the CMS experiment at the Large Hadron Collider can be useful in pre-training foundation models for HEP. Specifically, we introduce the AspenOpenJets dataset, consisting of approximately 178M high $p_T$ jets derived from CMS 2016 Open Data. We show how pre-training the OmniJet-$α$ foundation model on AspenOpenJets improves performance on generative tasks with significant domain shift: generating boosted top and QCD jets from the simulated JetClass dataset. In addition to demonstrating the power of pre-training of a jet-based foundation model on actual proton-proton collision data, we provide the ML-ready derived AspenOpenJets dataset for further public use.

Figures (4)

• Figure 1: Comparison of AspenOpenJets (AOJ) and QCD JetClass distributions for particle and jet level observables. Here $d_0$ and $d_z$ represent the transverse and longitudinal impact parameter respectively, with $\sigma_{d_0}$ and $\sigma_{d_z}$ being their uncertainties, and $\tau_{21}$ and $\tau_{32}$ are N-subjettiness variables. The last panel shows the average number of jet constituents for each particle species, accompanied by the corresponding $1\sigma$ standard deviation bands.
• Figure 2: The particle-level positional distribution of the AspenOpenJets in the $\eta-\phi$ plane, illustrating the granularity of the CMS detector. Note the appearance of vertical features indicating the endcaps at $|\eta| \sim 1.6$ and multiple "dead cells" throughout the detector.
• Figure 3: A comparison of the generation quality of models trained on QCD jets from JetClass, for different training sample sizes $D$. The performance of a foundation model pre-trained on AOJ (far left) is compared to a model trained from scratch (center left). The generation quality is compared across several high level features of the jets. Two quantitative metrics, the Kullback-Leibler divergence (center right) and Wasserstein-1 distance (far right), are computed as a function of the training sample size $D$ to compare how well the generated jets from each model matches the target distribution from JetClass. We also report the mean value and the envelopes over 5 trainings with different random seeds. When applicable, we also show power-law fits $\propto D^{-\gamma}$ to the metrics.
• Figure 4: A comparison of the generation quality of models trained on top-quark jets from JetClass, for different training sample sizes $D$. The performance of a foundation model pre-trained on AOJ (far left) is compared to a model trained from scratch (center left). The generation quality is compared across several high level features of the jets. Two quantitative metrics, the Kullback-Leibler divergence (center right) and Wasserstein-1 distance (far right), are computed as a function of the training sample size $D$ to compare how well the generated jets from each model matches the target distribution from JetClass. We also report the mean value and the envelopes over 5 trainings with different random seeds. When applicable, we also show power-law fits $\propto D^{-\gamma}$ to the metrics.