Autonomy Oriented Digital Twins for Real2Sim2Real Autoware Deployment

TL;DR

This work addresses the challenge of bridging real-world autonomous vehicle operation with high-fidelity simulation by developing autonomy-oriented digital twins across multiple scales and operational domains, and by tightly integrating the Autoware stack with the AutoDRIVE Ecosystem. The authors build vehicle and environment digital twins, calibrate them against real-world data, and establish cross-platform APIs/HMI frameworks to enable real2sim2real deployments, including the first off-road Autoware demonstration. Key contributions include a modular multi-scale vehicle model suite with detailed powertrain, suspension, tire, and sensor dynamics, a suite of environment twins for varied ODDs, and novel meta-packages within Autoware to handle diverse inputs/outputs across platforms. The work demonstrates end-to-end sim2real/autoware demonstrations on Nigel, F1TENTH, Hunter SE, and OpenCAV platforms, highlighting practical pathways for rapid Autoware deployment and extended ODDs in both on-road and off-road contexts, with clear directions for future multi-agent and dynamic-replanning extensions.

Abstract

Modeling and simulation of autonomous vehicles plays a crucial role in achieving enterprise-scale realization that aligns with technical, business and regulatory requirements. Contemporary trends in digital lifecycle treatment have proven beneficial to support SBD as well as V&V of these complex systems. Although, the development of appropriate fidelity simulation models capable of capturing the intricate real-world physics and graphics (real2sim), while enabling real-time interactivity for decision-making, has remained a challenge. Nevertheless, recent advances in AI-based tools and workflows, such as online deep-learning algorithms leveraging live-streaming data sources, offer the tantalizing potential for real-time system-identification and adaptive modeling to simulate vehicles, environments, as well as their interactions. This transition from virtual prototypes to digital twins not only improves simulation fidelity and real-time factor, but can also support the development of online adaption/augmentation techniques that can help bridge the gap between simulation and reality (sim2real). In such a milieu, this work focuses on developing autonomy-oriented digital twins of vehicles across different scales and configurations to help support the streamlined development and deployment of Autoware stack, using a unified real2sim2real toolchain. Particularly, the core deliverable for this project was to integrate the Autoware stack with AutoDRIVE Ecosystem to demonstrate end-to-end task of map-based autonomous navigation. This work discusses the development of vehicle and environment digital twins using AutoDRIVE Ecosystem, along with various APIs and HMIs to connect with the same, followed by a detailed section on AutoDRIVE-Autoware integration. Furthermore, this study describes the first-ever off-road deployment of the Autoware stack, expanding the ODD beyond on-road autonomous navigation.

Figures (40)

• Figure 1: Project overview demonstrating the integration of AutoDRIVE Ecosystem with Autoware stack for the application of autonomous valet parking using the OpenCAV.
• Figure 2: Project tools: AutoDRIVE Ecosystem employed to develop autonomy-oriented digital twins of small-scale (Nigel and F1TENTH), mid-scale (Hunter SE) and full-scale (OpenCAV) vehicles for deploying the Autoware stack.
• Figure 3: Project timeline Gantt chart.
• Figure 4: Autonomy-oriented vehicle digital twins across scales: Nigel and F1TENTH (small-scale), Husky and Hunter SE (mid-scale), and OpenCAV and RZR (full-scale) platforms for on/off-road autonomy.
• Figure 5: Simulation of vehicle dynamics, sensors and actuators for Nigel and F1TENTH digital twins.