Autonomous Pressure Control in MuVacAS via Deep Reinforcement Learning and Deep Learning Surrogate Models

TL;DR

Problem: maintaining precise argon-driven pressure in MuVacAS for fusion-material testing. Approach: a data-driven pipeline with a Fourier Neural Operator surrogate to emulate gas dynamics, paired with Proximal Policy Optimization reinforcement learning within a fast digital twin. Findings: the DRL policy can reach and hold target pressures under disturbances in both simulation and on the real prototype, even with changes in injector placement, and outperforms PID baselines. Significance: demonstrates a viable AI-driven approach for autonomous control of non-linear vacuum and gas injection subsystems in future fusion facilities and highlights the value of combining surrogate modeling with reinforcement learning for safe, efficient real-world deployment.

Abstract

The development of nuclear fusion requires materials that can withstand extreme conditions. The IFMIF-DONES facility, a high-power particle accelerator, is being designed to qualify these materials. A critical testbed for its development is the MuVacAS prototype, which replicates the final segment of the accelerator beamline. Precise regulation of argon gas pressure within its ultra-high vacuum chamber is vital for this task. This work presents a fully data-driven approach for autonomous pressure control. A Deep Learning Surrogate Model, trained on real operational data, emulates the dynamics of the argon injection system. This high-fidelity digital twin then serves as a fast-simulation environment to train a Deep Reinforcement Learning agent. The results demonstrate that the agent successfully learns a control policy that maintains gas pressure within strict operational limits despite dynamic disturbances. This approach marks a significant step toward the intelligent, autonomous control systems required for the demanding next-generation particle accelerator facilities.

Figures (7)

• Figure 1: Evaluation of the DRL agent on the real prototype. Top: achieved and reference pressures. Middle: absolute error between the achieved and reference pressures. Bottom: argon injection values. Here $p_{target}$ denotes the pressure in the lattice section where the collision chamber is located.
• Figure 2: Iterative evaluation through time of the FNO surrogate model. It takes an initial pressure distribution along the accelerator longitude (in arbitrary units, each point represents a sensor) and different argon injections in each step. The vertical axis in each plot represents $z$, the colorbar represents the pressure $p$ and the horizontal axis is the time $t$. From top to bottom: the real pressure distributions from the test dataset, the simulated by the model pressure distributions and the residual errors.
• Figure 3: Policy neural network. Its inputs and output are described in Table \ref{['tab:env_spaces']}.
• Figure 4: Evaluation of the DRL agent on a different simulated environment. Top: observed (achieved by the agent) and objective (reference) pressures. Bottom: argon injection values.
• Figure 5: Evaluation of the DRL agent on a different simulated environment. Top: observed (achieved by the agent) and objective (reference) pressures. Bottom: argon injection values.