Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Adaptive Data-Enabled Policy Optimization Approach for Autonomous Bicycle Control

Niklas Persson, Feiran Zhao, Mojtaba Kaheni, Florian Dörfler, Alessandro V. Papadopoulos

TL;DR

This paper addresses balancing a nonlinear, open-loop unstable autonomous bicycle by combining a stabilizing inner Feedback Linearization (FL) controller with an adaptive outer Data-Enabled Policy Optimization (DeePO) controller. DeePO learns and updates the LQR gains directly from online data using a covariance parameterization with exponentially weighted data and a forgetting factor, while also incorporating a robustness-promoting regularizer to obtain a stabilizing initial policy. The approach is validated through high-fidelity simulations and hardware experiments on an instrumented bicycle, showing improved tracking of the lean angle and lean rate compared to FL alone, and demonstrating the practical feasibility of data-driven adaptive control for nonlinear, time-varying systems. The work highlights that updating the policy gains at every time step is not always necessary and that DeePO can adapt to real-world nonlinearities, disturbances, and hardware limitations, with implications for autonomous bicycle control and related robotic platforms.

Abstract

This paper presents a unified control framework that integrates a Feedback Linearization (FL) controller in the inner loop with an adaptive Data-Enabled Policy Optimization (DeePO) controller in the outer loop to balance an autonomous bicycle. While the FL controller stabilizes and partially linearizes the inherently unstable and nonlinear system, its performance is compromised by unmodeled dynamics and time-varying characteristics. To overcome these limitations, the DeePO controller is introduced to enhance adaptability and robustness. The initial control policy of DeePO is obtained from a finite set of offline, persistently exciting input and state data. To improve stability and compensate for system nonlinearities and disturbances, a robustness-promoting regularizer refines the initial policy, while the adaptive section of the DeePO framework is enhanced with a forgetting factor to improve adaptation to time-varying dynamics. The proposed DeePO+FL approach is evaluated through simulations and real-world experiments on an instrumented autonomous bicycle. Results demonstrate its superiority over the FL-only approach, achieving more precise tracking of the reference lean angle and lean rate.

An Adaptive Data-Enabled Policy Optimization Approach for Autonomous Bicycle Control

TL;DR

This paper addresses balancing a nonlinear, open-loop unstable autonomous bicycle by combining a stabilizing inner Feedback Linearization (FL) controller with an adaptive outer Data-Enabled Policy Optimization (DeePO) controller. DeePO learns and updates the LQR gains directly from online data using a covariance parameterization with exponentially weighted data and a forgetting factor, while also incorporating a robustness-promoting regularizer to obtain a stabilizing initial policy. The approach is validated through high-fidelity simulations and hardware experiments on an instrumented bicycle, showing improved tracking of the lean angle and lean rate compared to FL alone, and demonstrating the practical feasibility of data-driven adaptive control for nonlinear, time-varying systems. The work highlights that updating the policy gains at every time step is not always necessary and that DeePO can adapt to real-world nonlinearities, disturbances, and hardware limitations, with implications for autonomous bicycle control and related robotic platforms.

Abstract

This paper presents a unified control framework that integrates a Feedback Linearization (FL) controller in the inner loop with an adaptive Data-Enabled Policy Optimization (DeePO) controller in the outer loop to balance an autonomous bicycle. While the FL controller stabilizes and partially linearizes the inherently unstable and nonlinear system, its performance is compromised by unmodeled dynamics and time-varying characteristics. To overcome these limitations, the DeePO controller is introduced to enhance adaptability and robustness. The initial control policy of DeePO is obtained from a finite set of offline, persistently exciting input and state data. To improve stability and compensate for system nonlinearities and disturbances, a robustness-promoting regularizer refines the initial policy, while the adaptive section of the DeePO framework is enhanced with a forgetting factor to improve adaptation to time-varying dynamics. The proposed DeePO+FL approach is evaluated through simulations and real-world experiments on an instrumented autonomous bicycle. Results demonstrate its superiority over the FL-only approach, achieving more precise tracking of the reference lean angle and lean rate.

Paper Structure

This paper contains 20 sections, 1 theorem, 29 equations, 13 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Statement of Contributions
  3. Organization
  4. Notation
  5. Autonomous bicycle control
  6. Bicycle Dynamics and Mathematical Modeling
  7. Control overview
  8. Data-enabled policy optimization for autonomous bicycle control
  9. The linear quadratic regulator
  10. Data-driven covariance parametrization of the LQR with exponential weighted data
  11. Data-enabled policy optimization for adaptive LQR control with exponentially weighted data
  12. Learning an initial stabilizing policy using robustness promoting regularization
  13. Control gain update rate
  14. Simulations and experiments
  15. Instrumented Bicycle
  16. ...and 5 more sections

Key Result

Lemma 1

For $V\in \mathcal{S}_t$, the gradient of $J_t(V)$ with respect to $V$ is given by where $P_t$ satisfies the Lyapunov equation

Figures (13)

  • Figure 1: Illustration of the parameters used in the bicycle model in \ref{['eq:simpleModel']}.
  • Figure 2: Stable and unstable regions of the instrumented bicycle, with the weave speed and capsize speed denoted by $v_w$ and $v_c$, respectively.
  • Figure 3: Control overview
  • Figure 4: Instrumented bicycle used in the experiments.
  • Figure 5: Control setup where the gray box represents the system controlled using DeePO.
  • ...and 8 more figures

Theorems & Definitions (3)

  • Lemma 1: zhao2024data
  • Remark 1
  • Remark 2