Object Importance Estimation using Counterfactual Reasoning for Intelligent Driving

TL;DR

This work tackles object importance estimation in a data-driven fashion and introduces HOIST - Human-annotated Object Importance in Simulated Traffic and proposes a novel approach that relies on counterfactual reasoning to estimate an object's importance.

Abstract

The ability to identify important objects in a complex and dynamic driving environment is essential for autonomous driving agents to make safe and efficient driving decisions. It also helps assistive driving systems decide when to alert drivers. We tackle object importance estimation in a data-driven fashion and introduce HOIST - Human-annotated Object Importance in Simulated Traffic. HOIST contains driving scenarios with human-annotated importance labels for vehicles and pedestrians. We additionally propose a novel approach that relies on counterfactual reasoning to estimate an object's importance. We generate counterfactual scenarios by modifying the motion of objects and ascribe importance based on how the modifications affect the ego vehicle's driving. Our approach outperforms strong baselines for the task of object importance estimation on HOIST. We also perform ablation studies to justify our design choices and show the significance of the different components of our proposed approach.

Figures (5)

• Figure 1: We perform counterfactual reasoning to identify important vehicles in a driving scenario. We modify the motion of vehicles, and ascribe importance based on how the modification affects the ego vehicle's driving.
• Figure 2: The removal score is the L2 distance between the corresponding waypoints of the true trajectory (shown in a) and the predicted trajectory after removing Car 0 (shown in b).
• Figure 3: The velocity perturbation score assesses potential collision, when the non-ego vehicle undergoes a velocity perturbation. A collision occurs when the corresponding waypoints of the ego vehicle and the non-ego vehicle trajectories are within a threshold. The collisions occur at the $k^{th}$ waypoint. The trajectories of the ego vehicle and the non-ego vehicle are shown in in orange and green, respectively.
• Figure 4: Qualitative comparison of our approach with the baselines. The objects are marked as important based on the optimal threshold for the F1 Score for each approach.
• Figure 5: Failure cases of our method.