Hybrid Modeling, Sim-to-Real Reinforcement Learning, and Large Language Model Driven Control for Digital Twins

TL;DR

This work evaluates a digital twin pipeline for a miniature greenhouse by comparing physics-based, data-driven, and hybrid HAM predictors across interpolation and extrapolation, and by testing Model Predictive Control, Reinforcement Learning, and Large Language Model-based controllers. HAM emerges as the best overall predictor in balance between accuracy, generalization, and computation, while LSTM achieves the highest precision at greater resource cost. Among controllers, MPC offers robust, predictable performance; RL demonstrates strong adaptability; LLM-based controllers provide flexible human–AI interaction when integrated with predictive surrogates. The study demonstrates sim-to-real transfer and discusses the practical implications of integrating LLMs for interpretable, tool-assisted control in digital twins, outlining directions for open-source deployment and broader applicability.

Abstract

This work investigates the use of digital twins for dynamical system modeling and control, integrating physics-based, data-driven, and hybrid approaches with both traditional and AI-driven controllers. Using a miniature greenhouse as a test platform, four predictive models Linear, Physics-Based Modeling (PBM), Long Short Term Memory (LSTM), and Hybrid Analysis and Modeling (HAM) are developed and compared under interpolation and extrapolation scenarios. Three control strategies Model Predictive Control (MPC), Reinforcement Learning (RL), and Large Language Model (LLM) based control are also implemented to assess trade-offs in precision, adaptability, and implementation effort. Results show that in modeling HAM provides the most balanced performance across accuracy, generalization, and computational efficiency, while LSTM achieves high precision at greater resource cost. Among controllers, MPC delivers robust and predictable performance, RL demonstrates strong adaptability, and LLM-based controllers offer flexible human-AI interaction when coupled with predictive tools.

Figures (18)

• Figure 1: Asset
• Figure 2: Architecture for model experiments using the prediction models. For each timestep $t$, the models get the previous $n$ states $\bm{x}$ and the previous and future $n$ controls $\bm{u}$. When the model makes a prediction for timestep $t+1$, it inserts the prediction for timestep $t+1$ into the list of previous measurements before making the next prediction.
• Figure 3: Structure for the experiments using the controllers. The controllers have a reference value for the state $\bm{x_{ref}}$, and they get the current state $\bm{x}$ from the sensors. The controllers will send the desired inputs $\bm{u}$ to the actuators, that will affect the temperature in the Greenhouse.
• Figure 4: Schematic representation of the MPC. The MPC consists of a minimizer that determines the optimal control inputs $\bm{u_t}$ to minimize the cost function. The linear model predicts the current state $\bm{x_t}$ based on the previous state $\bm{x_{t-1}}$ and control inputs $\bm{u_t}$. The cost function evaluates the performance by computing the deviation between the predicted state and the reference state $\bm{x_{ref}}$, as well as the effort associated with the control actions.
• Figure 5: The composition of the RL controller. It consists of a Deep Q-learning Network algorithm that chooses actions $\bm{u}$ given the current state $\bm{x}$, and the environment that the actions are applied to. The environment will perform a step for each action, that returns the next state. The reward will be caluclated based on the state $\bm{x}$, action $\bm{u}$ and reference state $\bm{x_{ref}}$ in the environment.