Modelling, Positioning, and Deep Reinforcement Learning Path Tracking Control of Scaled Robotic Vehicles: Design and Experimental Validation

TL;DR

The paper addresses indoor positioning and path tracking for scaled robotic cars by combining sensor fusion with a federated EKF and a DRL based path tracking controller trained with an expert demonstrator. A two-local EKF plus master fusion provides robust, high-rate position estimates while handling sensor faults and varying sampling rates. A novel DRL path tracking framework uses a digital twin for training and an expert-guided reward to mitigate sim-to-real gaps, achieving performance competitive with or better than model-based controllers. Experimental validation on the Quanser QCar demonstrates improved localization accuracy and effective, robust closed-loop path tracking across unseen trajectories. The work offers a practical, replicable approach for robust autonomous operation of small-scale mobile robots in indoor settings.

Abstract

Mobile robotic systems are becoming increasingly popular. These systems are used in various indoor applications, raging from warehousing and manufacturing to test benches for assessment of advanced control strategies, such as artificial intelligence (AI)-based control solutions, just to name a few. Scaled robotic cars are commonly equipped with a hierarchical control acthiecture that includes tasks dedicated to vehicle state estimation and control. This paper covers both aspects by proposing (i) a federeted extended Kalman filter (FEKF), and (ii) a novel deep reinforcement learning (DRL) path tracking controller trained via an expert demonstrator to expedite the learning phase and increase robustess to the simulation-to-reality gap. The paper also presents the formulation of a vehicle model along with an effective yet simple procedure for identifying tis paramters. The experimentally validated model is used for (i) supporting the design of the FEKF and (ii) serving as a digital twin for training the proposed DRL-based path tracking algorithm. Experimental results confirm the ability of the FEKF to improve the estimate of the mobile robot's position. Furthermore, the effectiveness of the DRL path tracking strateguy is experimentally tested along manoeuvres not considered during training, showing also the ability of the AI-based solution to outpeform model-based control strategies and the demonstrator. The comparison with benchmraking controllers is quantitavely evalueted through a set of key performance indicators.

Figures (22)

• Figure 1: Quanser QCar picture with sensors [38]
• Figure 2: Base station and QCar used as experimental platform.
• Figure 3: Reference path and relative position and heading errors.
• Figure 4: Experimental validation of the steady-state longitudinal dynamics: identification data set (blue square), validation data set (red diamond), and optimised steady-state speed regression model (green solid line).
• Figure 5: Experimental validation of the transient longitudinal dynamics: experimental data (blue solid line) and response of the optimised model \ref{['eq:motor_dyn']} (red solid line) when the step inputs are (a) $V_a=$ 1.25 V , (b) $V_a=$ 1.5 V , (c) $V_a=$ 1.1 V, and (d) $V_a=$ 1.35 V.