GPT-like transformer model for silicon tracking detector simulation

TL;DR

The paper investigates a GPT-like decoder-only transformer to generate silicon tracking detector hits as a sequence, aiming to replace parts of the Geant4 simulation with a fast, generative method. By tokenizing per-hit features and using sliding-window attention, the approach preserves correlations between hits and achieves muon-level tracking performance comparable to full simulation on the Open Data Detector. Electron and pion predictions reveal limitations from rare processes and tokenisation granularity, guiding future improvements such as weighting and secondary-particle modeling. GPU-based inference shows potential for production-scale speedups, though the method remains constrained by iterative generation and discrete outputs. The work provides open-source tools (SiliconAI) and reproducible datasets to enable further development in fast silicon-tracking simulations.

Abstract

Simulating physics processes and detector responses is essential in high energy physics and represents significant computing costs. Generative machine learning has been demonstrated to be potentially powerful in accelerating simulations, outperforming traditional fast simulation methods. The efforts have focused primarily on calorimeters. This work presents the very first studies on using neural networks for silicon tracking detectors simulation. The GPT-like transformer architecture is determined to be optimal for this task and applied in a fully generative way, ensuring full correlations between individual hits. Taking parallels from text generation, hits are represented as a flat sequence of feature values. The resulting tracking performance, evaluated on the Open Data Detector, is comparable with the full simulation.

Figures (10)

• Figure 1: A schematic illustration of a muon track in a two-component silicon tracking detector system in a transverse plane. The light and dark blue lines represent sensitive detectors of two types. The orange curved line represents a muon track that scatters on a detector element
• Figure 2: Graphical illustration of the track hits data representation in a 3D way, as in the output of the simulation (top) and a 2D flattened way, as used in the transformer model (bottom). For simplicity three features per hit are used in the illustration
• Figure 3: Illustration of the sliding window attention training. Individual full sequences are split in sliding windows of multiple hits. Models are then trained on individual windows. For simplicity three features per hit and two hits per window are used in the illustration
• Figure 4: Comparisons of numbers of simulated hits of single $\mu^{-}$ particles simulated with Geant4 (red) and the neural network (blue). The total number of hits (left) and the difference in total number of hits (right) are shown
• Figure 5: Comparisons of simulated hit properties of single $\mu^{-}$ particles simulated with Geant4 (red) and the neural network (blue). Surface coordinate $x$ of the hit (left) and global hit coordinate $z$ (right) are shown