AI-Driven approach for sustainable extraction of earth's subsurface renewable energy while minimizing seismic activity

TL;DR

The paper addresses human-induced seismicity during underground energy extraction by coupling a robust region-based seismicity-rate tracker with gain-scheduled reinforcement learning (DDPG) to tune a Multi-Input-Multi-Output Super-Twisting controller for injection control. It formulates region-wise SR dynamics via a diffusion model $u_t= c_{hy}\nabla^2 u + \frac{1}{\beta}\langle \bar{\mathcal{B}}_c, \bar{Q}_c\rangle$ and tracking of $y_i=\ln(R_i)$ to references $r(t)$ while enforcing flux-restriction constraints. The main contributions are (i) a control-RL framework that adapts $K_1$, $K_2$, and $l$ in real time, (ii) a model-free RL setup (DDPG) with a reward that trades off tracking accuracy and energy use, and (iii) numerical simulations showing improved SR tracking, reduced actuator energy, and confinement of high-pressure/Seismicity-rate regions under varying demand. This approach enables more sustainable subsurface energy deployment by mitigating seismic risk despite parametric uncertainties and unmodeled dynamics.

Abstract

Deep Geothermal Energy, Carbon Capture and Storage, and Hydrogen Storage hold considerable promise for meeting the energy sector's large-scale requirements and reducing CO$_2$ emissions. However, the injection of fluids into the Earth's crust, essential for these activities, can induce or trigger earthquakes. In this paper, we highlight a new approach based on Reinforcement Learning for the control of human-induced seismicity in the highly complex environment of an underground reservoir. This complex system poses significant challenges in the control design due to parameter uncertainties and unmodeled dynamics. We show that the reinforcement learning algorithm can interact efficiently with a robust controller, by choosing the controller parameters in real-time, reducing human-induced seismicity and allowing the consideration of further production objectives, \textit{e.g.}, minimal control power. Simulations are presented for a simplified underground reservoir under various energy demand scenarios, demonstrating the reliability and effectiveness of the proposed control-reinforcement learning approach.

Figures (10)

• Figure 1: Underground reservoir diagram (see also b:Gutierrez-Stefanou-2024).
• Figure 2: a) Regions $V_1$ and $V_2$ and location of the injection well with flux $Q_{s_1}$ inside region $V_2$. b) Seismicity rate in both regions, $V_1,V_2$ with constant injection rate, $Q_{s_1}=15$ [m$^3$/hr]. $1100$ more earthquakes of a given magnitude in a given time window are expected over the outer region of the reservoir due to the constant fluid injection.
• Figure 3: Solution, $u(x,t)$, of the pressure's reservoir at different times, with constant injection rate, $Q_{s_1}=15$ [m$^3$/hr]. No control is applied. The solution presents high-pressure profiles in wide areas next to the injection point. Observing the contour lines, the steady state is reached after two years.
• Figure 4: Schematic representation of the control-RL strategy for minimizing induced seismicity over an underground reservoir.
• Figure 5: Regions $V_1$ and $V_2$ and location of the injection wells.