Does Knowledge About Perceptual Uncertainty Help an Agent in Automated Driving?

TL;DR

The paper investigates how perceptual uncertainty affects reinforcement learning for automated driving and whether exposing uncertainty information to the agent improves performance. Using a CARLA-based PPO setup with BEV semantic segmentation perturbations, it compares learning under correct perception, perturbed perception without uncertainty information, and perturbed perception with uncertainty information in the observation. Results show that unreliable perception alone induces defensive driving, while including uncertainty signals enables the agent to adapt its behavior to the current situation and complete tasks more efficiently while accounting for risk. The work provides a concrete experimental framework for studying RL under perceptual uncertainty and demonstrates the practical value of uncertainty-aware observations in autonomous driving.

Abstract

Agents in real-world scenarios like automated driving deal with uncertainty in their environment, in particular due to perceptual uncertainty. Although, reinforcement learning is dedicated to autonomous decision-making under uncertainty these algorithms are typically not informed about the uncertainty currently contained in their environment. On the other hand, uncertainty estimation for perception itself is typically directly evaluated in the perception domain, e.g., in terms of false positive detection rates or calibration errors based on camera images. Its use for deciding on goal-oriented actions remains largely unstudied. In this paper, we investigate how an agent's behavior is influenced by an uncertain perception and how this behavior changes if information about this uncertainty is available. Therefore, we consider a proxy task, where the agent is rewarded for driving a route as fast as possible without colliding with other road users. For controlled experiments, we introduce uncertainty in the observation space by perturbing the perception of the given agent while informing the latter. Our experiments show that an unreliable observation space modeled by a perturbed perception leads to a defensive driving behavior of the agent. Furthermore, when adding the information about the current uncertainty directly to the observation space, the agent adapts to the specific situation and in general accomplishes its task faster while, at the same time, accounting for risks.

Figures (9)

• Figure 1: An illustration of the main idea of our experiments in the CARLA driving simulator. An agent perceives its environment through a semantic segmentation mask in a bird's eye view, which we perturb in a controlled manner. This corresponds to perfectly quantifiable perceptual uncertainties, which we can provide to the agent. In our study, we investigate whether the agent benefits from this uncertainty information.
• Figure 2: We consider four scenarios differing in perturbation of the observation space as well as the availability of the uncertainty information. Green color indicates the scenario-dependent components of the observation space.
• Figure 3: Observation space setup. The observation space dimension increases when the uncertainty is added. We distinguish between three components: 1) Vision component -- consisting of a flattened and gray-scaled semantic segmentation image; 2) Non-visual component - containing, e.g., velocity; 3) Uncertainty component -- containing the one-hot encoded uncertainty information.
• Figure 4: A summary of the perceptual perturbations used in our experiments. Four different perturbations are used for simulating uncertainty in perception. The last case stands for a correct perception. We use the abbreviations for visible (V), agent/ego-vehicle (E) and invisible (X).
• Figure 5: Top down view of the agent's task route.