Agentic Physical AI toward a Domain-Specific Foundation Model for Nuclear Reactor Control

TL;DR

This work reframes nuclear reactor control as a data-driven, domain-specific foundation-model problem by introducing Agentic Physical AI: a compact 360M-parameter language model trained offline on synthetic reactor data to generate admissible actuation sequences validated through physics-based simulation. A two-phase curriculum (grammar learning then task grounding) combined with LoRA adaptation and large-scale data scaling yields a phase transition from high-variance imitation to stable, low-variance, outcome-driven control, reducing tail risk by orders of magnitude without online exploration or reward shaping. The approach demonstrates agentic behavior (policy optimization over multiple valid strategies), Physical AI (outcome-centric validation within deterministic reactor physics), and early signs of foundation-model properties (scaling-induced emergence and cross-simulator transfer to PyRK). This framework offers a practical pathway toward safe, reusable domain-specific intelligence for safety-critical cyber-physical systems, with implications for broader adoption beyond nuclear control. The work also outlines limitations and a roadmap to full domain coverage, including multi-step procedures, diverse reactor types, and uncertainty quantification, to realize complete domain-specific foundation models.

Abstract

The prevailing paradigm in AI for physical systems, scaling general-purpose foundation models toward universal multimodal reasoning, confronts a fundamental barrier at the control interface. Recent benchmarks show that even frontier vision-language models achieve only 50-53% accuracy on basic quantitative physics tasks, behaving as approximate guessers that preserve semantic plausibility while violating physical constraints. This input unfaithfulness is not a scaling deficiency but a structural limitation. Perception-centric architectures optimize parameter-space imitation, whereas safety-critical control demands outcome-space guarantees over executed actions. Here, we present a fundamentally different pathway toward domain-specific foundation models by introducing compact language models operating as Agentic Physical AI, in which policy optimization is driven by physics-based validation rather than perceptual inference. We train a 360-million-parameter model on synthetic reactor control scenarios, scaling the dataset from 10^3 to 10^5 examples. This induces a sharp phase transition absent in general-purpose models. Small-scale systems exhibit high-variance imitation with catastrophic tail risk, while large-scale models undergo variance collapse exceeding 500x reduction, stabilizing execution-level behavior. Despite balanced exposure to four actuation families, the model autonomously rejects approximately 70% of the training distribution and concentrates 95% of runtime execution on a single-bank strategy. Learned representations transfer across distinct physics and continuous input modalities without architectural modification.

Figures (9)

• Figure 1: Integrated framework for Agentic Physical AI in nuclear reactor control. Three interconnected paradigms form a coherent system: (Left) Agentic AI: the compact 360M-parameter model optimizes runtime policy away from balanced training distribution (KL divergence increases with scale: 0.18→0.31 nats), concentrating 76% of actions on single_b2 strategies despite only 30% training frequency, and deploying brittle multi-bank coordination sparingly (3.75%). (Center) Physical AI: success is defined by closed-loop execution in KOMODO simulator achieving target power within tolerance bands ($\pm$1--10%), not by parameter proximity to labels; the model learns to navigate the physics-constrained feasible manifold $\mathcal{M}_{\text{feas}}$ through 2,000 independent validation runs per scale. (Right) Foundation Model: scaling from 1K to 100K scenarios drives qualitative phase transitions: sub-1% precision jumps from 26.2% to 92% (super-linear), variance collapses 500×, policy entropy decreases from 1.38 to 0.89 nats despite increasing capacity (structural compression), and the model transfers to PyRK point kinetics with $>$94% success. The two-phase curriculum (Phase 1: grammar learning via CPT; Phase 2: task conditioning via LoRA) separates domain structure from task specialization, enabling reusable priors. Data scaling provides the feedback loop that stabilizes agentic policies through outcome-centric validation.
• Figure 2: Experimental workflow for Agentic Physical AI in nuclear reactor control. The pipeline consists of three stages: (Stage 1) Data generation: KOMODO simulator generates synthetic corpora at three scales (1K, 10K, 100K) with balanced actuation families (single-bank, simultaneous, sequential) to prevent trivial dataset bias. (Stage 2) Two-phase curriculum training: SmolLM2-360M backbone undergoes Phase 1 continued pretraining (CPT) to learn control command grammar without power conditioning, followed by Phase 2 supervised LoRA fine-tuning to map power targets to six-parameter rod commands. (Stage 3) Physical AI validation: trained models generate control proposals validated through 2,000 independent closed-loop KOMODO simulations per scale, with success measured by terminal power accuracy across five tolerance bands (plus or minus 1, 2, 3, 5, 10%), not by parameter proximity to labels. This outcome-centric evaluation enables agentic behavior by allowing the model to select among multiple admissible solutions within the physics-constrained feasible manifold.
• Figure 3: Scaling of validation success and regime robustness with dataset size.a. Validation success rates across tolerance bands show a sharp improvement between 10K and 100K, revealing the emergence of a stable, high-precision control policy, with sub-1% accuracy jumping from 26.2 to 92%. b. Performance stratified by power-change bins shows that the 100K model achieves regime-consistent precision that is absent in the 1K and 10K variants.
• Figure 4: Terminal power error distributions across dataset scales. a. CDF curves highlight tail-risk collapse at 100K scale. b. Violin plots demonstrate narrowing uncertainty and emergence of a stable, low-variance control policy.
• Figure 5: Benchmarking Agentic AI against classical PID and direct learning baselines.a. Overall success rates (plus or minus 5% tolerance). The Proposed (100K) model achieves 97.4% success, significantly outperforming the naive PID baseline (43.8%) which is limited by single-bank saturation. b. Success rates stratified by power change magnitude. PID performance collapses in the Large regime due to physical limits, whereas the Agentic AI maintains high reliability. c. Error distribution (log scale). While PID shows high precision (low median), it suffers from extreme outliers (high max error). The Agentic AI eliminates these catastrophic tails. d. Cumulative distribution function (CDF) of errors, highlighting the superior tail-risk management of the proposed model.