Reinforcement Learning-based Model Predictive Control for Greenhouse Climate Control

TL;DR

This work tackles greenhouse climate control under parametric uncertainty by marrying model predictive control with reinforcement learning. It introduces a parametrized MPC that serves as both policy provider and value function approximator, and learns its parameters online using second-order LSTD Q-learning, while enforcing explicit constraints through slack variables. In simulation on a lettuce greenhouse model with real and perturbed weather profiles, the MPC–RL controller substantially reduces constraint violations and achieves efficient crop growth compared to robust MPC and model-free RL baselines, approaching the performance of an ideal MPC with perfect model knowledge. The approach offers interpretability through learned MPC parameters and demonstrates data-efficient, uncertainty-tolerant control suitable for practical greenhouse operation, with potential extension to other crops and experimental validation.

Abstract

Greenhouse climate control is concerned with maximizing performance in terms of crop yield and resource efficiency. One promising approach is model predictive control (MPC), which leverages a model of the system to optimize the control inputs, while enforcing physical constraints. However, prediction models for greenhouse systems are inherently inaccurate due to the complexity of the real system and the uncertainty in predicted weather profiles. For model-based control approaches such as MPC, this can degrade performance and lead to constraint violations. Existing approaches address uncertainty in the prediction model with robust or stochastic MPC methodology; however, these necessarily reduce crop yield due to conservatism and often bear higher computational loads. In contrast, learning-based control approaches, such as reinforcement learning (RL), can handle uncertainty naturally by leveraging data to improve performance. This work proposes an MPC-based RL control framework to optimize the climate control performance in the presence of prediction uncertainty. The approach employs a parametrized MPC scheme that learns directly from data, in an online fashion, the parametrization of the constraints, prediction model, and optimization cost that minimizes constraint violations and maximizes climate control performance. Simulations show that the approach can learn an MPC controller that significantly outperforms the current state-of-the-art in terms of constraint violations and efficient crop growth.

Figures (10)

• Figure 1: Diagram of the MPC-based RL architecture
• Figure 2: Last two days of weather profiles. The black profile is the original, and the red lines are five example perturbed profiles.
• Figure 3: Stage cost, TD error, and constraint violations over 100 growth cycles of training.
• Figure 4: Outputs over first and last growth cycle of training with upper (red lines) and lower (black lines) bounds.
• Figure 5: Parameter evolution, for the 4 most changed parameters, over 100 growth cycles of training. Note that model parameters $\theta_p$ have been normalized with respect to their true values.

Theorems & Definitions (2)

• Remark 1
• Remark 2