Quantitative evaluation of brain-inspired vision sensors in high-speed robotic perception

TL;DR

The paper addresses evaluating brain-inspired vision sensors (BVS) for high-speed robotic perception by introducing a unified, quantitative evaluation framework that compares event-based sensors (EVS) and the primitive-based Tianmouc. It provides cross-modality calibrations, imaging-quality metrics, and task benchmarks (corner detection and motion estimation) across varying rotational speeds and illumination. Key findings show EVS excels in sparse, high-speed conditions but struggles with clutter and event-rate saturation, while Tianmouc offers robust, global high-speed sensing across scenarios. The framework enables fair, standardized assessment and informs sensor choice and design for high-speed robotics.

Abstract

Perception systems in robotics encounter significant challenges in high-speed and dynamic conditions when relying on traditional cameras, where motion blur can compromise spatial feature integrity and task performance. Brain-inspired vision sensors (BVS) have recently gained attention as an alternative, offering high temporal resolution with reduced bandwidth and power requirements. Here, we present the first quantitative evaluation framework for two representative classes of BVSs in variable-speed robotic sensing, including event-based vision sensors (EVS) that detect asynchronous temporal contrasts, and the primitive-based sensor Tianmouc that employs a complementary mechanism to encode both spatiotemporal changes and intensity. A unified testing protocol is established, including crosssensor calibrations, standardized testing platforms, and quality metrics to address differences in data modality. From an imaging standpoint, we evaluate the effects of sensor non-idealities, such as motion-induced distortion, on the capture of structural information. For functional benchmarking, we examine task performance in corner detection and motion estimation under different rotational speeds. Results indicate that EVS performs well in highspeed, sparse scenarios and in modestly fast, complex scenes, but exhibits performance limitations in high-speed, cluttered settings due to pixel-level bandwidth variations and event rate saturation. In comparison, Tianmouc demonstrates consistent performance across sparse and complex scenarios at various speeds, supported by its global, precise, high-speed spatiotemporal gradient samplings. These findings offer valuable insights into the applicationdependent suitability of BVS technologies and support further advancement in this area.

Figures (8)

• Figure 1: (a) A typical high-speed robot platform. (b) Challenges of motion blur in the Tianmouc COP pathway, which shares the same motion blur issues as traditional cameras, when the robot in (a) is turning rapidly. (c) The high-speed temporal difference and (d) spatial difference sampling in the Tianmouc AOP pathway demonstrate significant potential for motion-blur-resistant robotic perception.
• Figure 2: Overview of our work. (a) Visualization of difference in data modalities of different sensors. (b) The experimental setup. (c) Two key quality factors for evaluation.
• Figure 3: Evaluation methods for imaging quality of BVS.
• Figure 4: Evaluation of motion blur using thickness as the indicator under (a) low illumination and (b) high illumination.
• Figure 5: Evaluation of structural information indicators under (a) low illumination and (b) high illumination.