SignEye: Traffic Sign Interpretation from Vehicle First-Person View

TL;DR

A traffic guidance assistant (TGA) scenario application is developed to re-explore the role of traffic signs in ADS as a complement to popular autonomous technologies (such as obstacle perception) and results demonstrate that the TGA can provide complementary information to ADS beyond existing popular autonomous technologies.

Abstract

Traffic signs play a key role in assisting autonomous driving systems (ADS) by enabling the assessment of vehicle behavior in compliance with traffic regulations and providing navigation instructions. However, current works are limited to basic sign understanding without considering the egocentric vehicle's spatial position, which fails to support further regulation assessment and direction navigation. Following the above issues, we introduce a new task: traffic sign interpretation from the vehicle's first-person view, referred to as TSI-FPV. Meanwhile, we develop a traffic guidance assistant (TGA) scenario application to re-explore the role of traffic signs in ADS as a complement to popular autonomous technologies (such as obstacle perception). Notably, TGA is not a replacement for electronic map navigation; rather, TGA can be an automatic tool for updating it and complementing it in situations such as offline conditions or temporary sign adjustments. Lastly, a spatial and semantic logic-aware stepwise reasoning pipeline (SignEye) is constructed to achieve the TSI-FPV and TGA, and an application-specific dataset (Traffic-CN) is built. Experiments show that TSI-FPV and TGA are achievable via our SignEye trained on Traffic-CN. The results also demonstrate that the TGA can provide complementary information to ADS beyond existing popular autonomous technologies.

Figures (5)

• Figure 1: Illustration of the SignEye, where the TSI-FPV part interprets sign units as structured descriptions and assigns them to different roads and lanes in egocentric relative position definition (EgoRPD), where "C" for current, "L" for left, "R" for right, and "A" for all lanes or roads. The TGA part combines the descriptions with vehicle attributes and a route graph to achieve traffic regulation assessment and direction navigation.
• Figure 2: TGA estimates the structured sign descriptions in EgoRPD the vehicle attribute and route graph for achieving traffic regulation assessment and direction navigation respectively.
• Figure 3: Overall pipeline of SignEye. It takes a road image from the vehicle's first-person view, and object (sign, lane, and road) regions as input to generate structured sign descriptions in EgoRPD and combines them with vehicle attributes and route graphs to achieve the TGA scenario, automatically.
• Figure 4: Workflow of the data engine for building the Traffic-CN.
• Figure 5: Visualization of the egocentric vehicle corresponding description in the TSI-FPV task.