Digital Twin-based Control Co-Design of Full Vehicle Active Suspensions via Deep Reinforcement Learning

TL;DR

This paper presents a digital twin–driven, multi-generation control co-design framework for full-vehicle active suspensions, integrating deep reinforcement learning with differentiable dynamics to co-optimize both suspension hardware and control policies. It addresses partial observability and environmental uncertainty by incorporating quantile-based uncertainty updating within a DT loop, enabling personalized optimization for distinct driving styles. The results demonstrate driver-specific suspension designs and significantly reduced actuator effort while preserving ride comfort and stability, highlighting the potential of DT-enabled CCD to produce self-improving, adaptive vehicle subsystems across lifecycles.

Abstract

Active suspension systems are critical for enhancing vehicle comfort, safety, and stability, yet their performance is often limited by fixed hardware designs and control strategies that cannot adapt to uncertain and dynamic operating conditions. Recent advances in digital twins (DTs) and deep reinforcement learning (DRL) offer new opportunities for real-time, data-driven optimization across a vehicle's lifecycle. However, integrating these technologies into a unified framework remains an open challenge. This work presents a DT-based control co-design (CCD) framework for full-vehicle active suspensions using multi-generation design concepts. By integrating automatic differentiation into DRL, we jointly optimize physical suspension components and control policies under varying driver behaviors and environmental uncertainties. DRL also addresses the challenge of partial observability, where only limited states can be sensed and fed back to the controller, by learning optimal control actions directly from available sensor information. The framework incorporates model updating with quantile learning to capture data uncertainty, enabling real-time decision-making and adaptive learning from digital-physical interactions. The approach demonstrates personalized optimization of suspension systems under two distinct driving settings (mild and aggressive). Results show that the optimized systems achieve smoother trajectories and reduce control efforts by approximately 43% and 52% for mild and aggressive, respectively, while maintaining ride comfort and stability. Contributions include: developing a DT-enabled CCD framework integrating DRL and uncertainty-aware model updating for full-vehicle active suspensions, introducing a multi-generation design strategy for self-improving systems, and demonstrating personalized optimization of active suspension systems for distinct driver types.

Figures (17)

• Figure 1: Illustration of the differences among a digital model, a digital shadow, and a digital twin.
• Figure 2: Diagram of reinforcement learning (RL), modified from sutton1998reinforcement.
• Figure 3: Comparison of (a) conventional DRL-based control with fixed hardware parameter $\mathbf{p}$, where the policy $\boldsymbol\pi$ is only dependent of $\mathbf{p}$, and (b) hardware as policy, where the policy $\boldsymbol\pi$ is dependent of $\mathbf{x}$ and $\mathbf{p}$ and the hardware parameter and the policy are co-optimized (modified from chen2020hardware).
• Figure 4: Simulation workflow for the full-vehicle active suspension system. The driving and road profiles are combined to compute wheel-level elevations and elevation rates, which serve as external disturbances to the dynamic model.
• Figure 5: Model of full vehicle with active suspension systems.