Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Ev-Edge: Efficient Execution of Event-based Vision Algorithms on Commodity Edge Platforms

Shrihari Sridharan, Surya Selvam, Kaushik Roy, Anand Raghunathan

TL;DR

Ev-Edge is proposed, a framework that contains three key optimizations to boost the performance of event-based vision algorithms on edge platforms that achieves 1.28x-2.05x improvements in latency and 1.23x-2.15x in energy over an all-GPU implementation on the NVIDIA Jetson Xavier AGX platform for single-task execution scenarios.

Abstract

Event cameras have emerged as a promising sensing modality for autonomous navigation systems, owing to their high temporal resolution, high dynamic range and negligible motion blur. To process the asynchronous temporal event streams from such sensors, recent research has shown that a mix of Artificial Neural Networks (ANNs), Spiking Neural Networks (SNNs) as well as hybrid SNN-ANN algorithms are necessary to achieve high accuracies across a range of perception tasks. However, we observe that executing such workloads on commodity edge platforms which feature heterogeneous processing elements such as CPUs, GPUs and neural accelerators results in inferior performance. This is due to the mismatch between the irregular nature of event streams and diverse characteristics of algorithms on the one hand and the underlying hardware platform on the other. We propose Ev-Edge, a framework that contains three key optimizations to boost the performance of event-based vision systems on edge platforms: (1) An Event2Sparse Frame converter directly transforms raw event streams into sparse frames, enabling the use of sparse libraries with minimal encoding overheads (2) A Dynamic Sparse Frame Aggregator merges sparse frames at runtime by trading off the temporal granularity of events and computational demand thereby improving hardware utilization (3) A Network Mapper maps concurrently executing tasks to different processing elements while also selecting layer precision by considering both compute and communication overheads. On several state-of-art networks for a range of autonomous navigation tasks, Ev-Edge achieves 1.28x-2.05x improvements in latency and 1.23x-2.15x in energy over an all-GPU implementation on the NVIDIA Jetson Xavier AGX platform for single-task execution scenarios. Ev-Edge also achieves 1.43x-1.81x latency improvements over round-robin scheduling methods in multi-task execution scenarios.

Ev-Edge: Efficient Execution of Event-based Vision Algorithms on Commodity Edge Platforms

TL;DR

Ev-Edge is proposed, a framework that contains three key optimizations to boost the performance of event-based vision algorithms on edge platforms that achieves 1.28x-2.05x improvements in latency and 1.23x-2.15x in energy over an all-GPU implementation on the NVIDIA Jetson Xavier AGX platform for single-task execution scenarios.

Abstract

Event cameras have emerged as a promising sensing modality for autonomous navigation systems, owing to their high temporal resolution, high dynamic range and negligible motion blur. To process the asynchronous temporal event streams from such sensors, recent research has shown that a mix of Artificial Neural Networks (ANNs), Spiking Neural Networks (SNNs) as well as hybrid SNN-ANN algorithms are necessary to achieve high accuracies across a range of perception tasks. However, we observe that executing such workloads on commodity edge platforms which feature heterogeneous processing elements such as CPUs, GPUs and neural accelerators results in inferior performance. This is due to the mismatch between the irregular nature of event streams and diverse characteristics of algorithms on the one hand and the underlying hardware platform on the other. We propose Ev-Edge, a framework that contains three key optimizations to boost the performance of event-based vision systems on edge platforms: (1) An Event2Sparse Frame converter directly transforms raw event streams into sparse frames, enabling the use of sparse libraries with minimal encoding overheads (2) A Dynamic Sparse Frame Aggregator merges sparse frames at runtime by trading off the temporal granularity of events and computational demand thereby improving hardware utilization (3) A Network Mapper maps concurrently executing tasks to different processing elements while also selecting layer precision by considering both compute and communication overheads. On several state-of-art networks for a range of autonomous navigation tasks, Ev-Edge achieves 1.28x-2.05x improvements in latency and 1.23x-2.15x in energy over an all-GPU implementation on the NVIDIA Jetson Xavier AGX platform for single-task execution scenarios. Ev-Edge also achieves 1.43x-1.81x latency improvements over round-robin scheduling methods in multi-task execution scenarios.
Paper Structure (12 sections, 3 equations, 8 figures, 2 tables)

This paper contains 12 sections, 3 equations, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Related Work
  4. Ev-Edge Framework
  5. Event2Sparse Frame Converter (E2SF)
  6. Dynamic Sparse Frame Aggregator (DSFA)
  7. Network Mapper (NMP)
  8. Mapping methodology
  9. Scheduling methodology and Latency Estimation
  10. Experimental Methodology
  11. Results
  12. Conclusion

Figures (8)

  • Figure 1: Popular input representation schemes employed by state-of-the-art event-based algorithms
  • Figure 2: Ev-Edge framework for improving performance of event-based vision tasks on heterogeneous edge platforms. Ev-Edge consists of three components, viz., Network Mapper, Event2SparseFrame converter and Dynamic Sparse Frame Aggregator integrated into the inference pipeline
  • Figure 3: Temporal event density of Indoorflying2 segment from MVSEC dataset mvsec
  • Figure 4: Dynamic Sparse Frame Aggregator (DSFA)
  • Figure 5: (a) Example multi-task input graph. Candidate generated from the input graph by assigning each node to a processing element and adding communication nodes between nodes, (b) Execution order of the example candidate
  • ...and 3 more figures