PCLA: A Framework for Testing Autonomous Agents in the CARLA Simulator
Masoud Jamshidiyan Tehrani, Jinhan Kim, Paolo Tonella
TL;DR
The paper addresses the challenge of reusing CARLA Leaderboard agents in custom test environments, where coupling to the Leaderboard codebase and CARLA version hinders experiments. It introduces PCLA, an open-source testing framework that deploys pretrained Leaderboard agents onto a vehicle in arbitrary CARLA environments and outputs per-frame action commands, decoupling from the Leaderboard codebase and version constraints. Key contributions include nine high-performing Leaderboard agents with 17 seeds, a uniform per-frame action interface, and route-to-CARLA GPS translation that enables flexible scenario testing. This framework enables researchers to evaluate and compare agents across custom scenarios and supports rapid test generation and adversarial testing in CARLA, promoting reproducible and scalable evaluations.
Abstract
Recent research on testing autonomous driving agents has grown significantly, especially in simulation environments. The CARLA simulator is often the preferred choice, and the autonomous agents from the CARLA Leaderboard challenge are regarded as the best-performing agents within this environment. However, researchers who test these agents, rather than training their own ones from scratch, often face challenges in utilizing them within customized test environments and scenarios. To address these challenges, we introduce PCLA (Pretrained CARLA Leaderboard Agents), an open-source Python testing framework that includes nine high-performing pre-trained autonomous agents from the Leaderboard challenges. PCLA is the first infrastructure specifically designed for testing various autonomous agents in arbitrary CARLA environments/scenarios. PCLA provides a simple way to deploy Leaderboard agents onto a vehicle without relying on the Leaderboard codebase, it allows researchers to easily switch between agents without requiring modifications to CARLA versions or programming environments, and it is fully compatible with the latest version of CARLA while remaining independent of the Leaderboard's specific CARLA version. PCLA is publicly accessible at https://github.com/MasoudJTehrani/PCLA.