Learning-Based Design of Off-Policy Gaussian Controllers: Integrating Model Predictive Control and Gaussian Process Regression

Abstract

This paper presents an off-policy Gaussian Predictive Control (GPC) framework aimed at solving optimal control problems with a smaller computational footprint, thereby facilitating real-time applicability while ensuring critical safety considerations. The proposed controller imitates classical control methodologies by modeling the optimization process through a Gaussian process and employs Gaussian Process Regression to learn from the Model Predictive Control (MPC) algorithm. Notably, the Gaussian Process setup does not incorporate a built-in model, enhancing its applicability to a broad range of control problems. We applied this framework experimentally to a differential drive mobile robot, tasking it with trajectory tracking and obstacle avoidance. Leveraging the off-policy aspect, the controller demonstrated adaptability to diverse trajectories and obstacle behaviors. Simulation experiments confirmed the effectiveness of the proposed GPC method, emphasizing its ability to learn the dynamics of optimal control strategies. Consequently, our findings highlight the significant potential of off-policy Gaussian Predictive Control in achieving real-time optimal control for handling of robotic systems in safety-critical scenarios.

Figures (9)

• Figure 1: Architecture of the proposed off-policy Gaussian Predictive Control framework, composed of three main components: 1) An On-Policy MPC controller as the foundational layer of the controller; 2) A Learning Process, where the system’s interaction with the environment is observed and are collected and a generalized learning model is trained; 3) An Off-Policy GPC controller, evolved and designed based on Gaussian Processes, that becomes the primary controller once it has adequately learned and evolved from the acquired data.
• Figure 2: Plot of trajectories: (a) Leminscate of Gerono; (b) Ellipse; (c) Cycloid; and (d) Sine wave.
• Figure 3: MPC performance on the robot to follow an elliptical reference trajectory while avoiding an obstacle trajectory that is a Leminscate of Geron.
• Figure 4: MPC performance on the robot to follow a sine wave reference trajectory while avoiding an elliptical obstacle trajectory.
• Figure 5: The controller was trained on the first half of the environment and tested on the second half. The plots depict the torque commands of the left and right wheels for both MPC and GPC controllers, and the variance computed by the GP regression.