Research, Applications and Prospects of Event-Based Pedestrian Detection: A Survey

TL;DR

Event-based cameras offer asynchronous, low-latency sensing with high dynamic range, enabling robust pedestrian detection in dynamic and challenging lighting. This survey synthesizes EB-PD approaches across three data-input paradigms (direct event streams, event-to-frame conversions, and parallel fusion) and surveys deep-learning and hybrid models, datasets, and evaluation standards. It highlights core datasets (e.g., PEDRo, GEN1, Prophesee, Neuromomo) and models adapted for event data (YOLO variants, GA-BP, SAM, SNNs), while candidly discussing real-world challenges such as dataset standardization, hardware costs, and the need for specialized event-driven architectures. The paper argues that advancing EB-PD will rely on standardized benchmarks, cost-effective edge hardware, and specialized models that exploit sparse, high-temporal-resolution data, with future directions in sense-compute convergence, adaptive cameras, and broader application horizons.

Abstract

Event-based cameras, inspired by the biological retina, have evolved into cutting-edge sensors distinguished by their minimal power requirements, negligible latency, superior temporal resolution, and expansive dynamic range. At present, cameras used for pedestrian detection are mainly frame-based imaging sensors, which have suffered from lethargic response times and hefty data redundancy. In contrast, event-based cameras address these limitations by eschewing extraneous data transmissions and obviating motion blur in high-speed imaging scenarios. On pedestrian detection via event-based cameras, this paper offers an exhaustive review of research and applications particularly in the autonomous driving context. Through methodically scrutinizing relevant literature, the paper outlines the foundational principles, developmental trajectory, and the comparative merits and demerits of eventbased detection relative to traditional frame-based methodologies. This review conducts thorough analyses of various event stream inputs and their corresponding network models to evaluate their applicability across diverse operational environments. It also delves into pivotal elements such as crucial datasets and data acquisition techniques essential for advancing this technology, as well as advanced algorithms for processing event stream data. Culminating with a synthesis of the extant landscape, the review accentuates the unique advantages and persistent challenges inherent in event-based pedestrian detection, offering a prognostic view on potential future developments in this fast-progressing field.

Figures (5)

• Figure 1: Event-based camera captures visual information through a process that mirrors the human eye: each pixel operates independently, like retinal photoreceptors, responding only to changes in light intensity. An 'ON event' is triggered by an increase, and an 'OFF event' by a decrease, mimicking the bipolar cells' response.
• Figure 2: Number and distribution of papers over the years. Number of papers The upward trend in the number of papers shows the increasing interest in EB-PD tasks. *Note that as of 28 February 2024, no relevant literature has been found for 2024 at this time.
• Figure 3: Methods for EB-PD are typically segmented into four distinct stages. These methodologies can be categorized based on the data input modality: Event-to-Frame Input Processing, Pure Event Stream Input, and Combined Event and Frame Input. The preprocessing of the signal is deemed a pivotal step across these types. Pure Event Stream Input focuses on the meticulous preprocessing and analysis of raw event data, capitalizing on the high temporal resolution of event streams. Event-to-Frame Input Processing underscores the algorithmic and feature processing nuances inherent in the data transformation process, adapting event streams into a frame-like structure for compatibility with conventional vision algorithms. The Combined Event and Frame Input seeks to amalgamate the strengths of both data forms, harnessing the high temporal resolution of event streams and the rich spatial information of traditional frames, thereby leveraging their complementary nature. Concurrently, we posit that EB-PD harbors substantial potential for diverse applications, given its unique capacity to capture and analyze dynamic visual information.
• Figure 4: The conversion of event streams to event frames is essentially the squashing of event signals within a time window onto a single frame, (a) specifying the luminance of a pixel point based on the frequency of the event's occurrence, (b) temporal ordering of the events based on when they occur within a selected time window, and (c) converting the number of events on a pixel point into the luminance value of an image by simulating the accumulation of neurons with potentials up to a threshold and then doling them out.
• Figure 5: GMVDT-NVS's equipment installation diagram, where the authors installed two DVS346s and two frame-based cameras on the roof of the car for road testing.