Real Time Scheduling Framework for Multi Object Detection via Spiking Neural Networks

TL;DR

This work tackles real-time, energy-efficient multi-object detection on autonomous mobile agents by leveraging spiking neural networks. It introduces RT-SNN, a framework that offers flexible execution options through adjustable timesteps and optional membrane potential reuse, paired with a membrane confidence metric to predict accuracy. The authors combine offline schedulability analysis with a runtime NPFP-based scheduler to maximize accuracy while preserving deadlines, and implement the approach on Spiking-YOLO. Experimental results on KITTI demonstrate that RT-SNN can meet timing guarantees, improve detection accuracy, and reduce energy consumption relative to baseline approaches, making SNN-based MOD viable for constrained platforms.

Abstract

Given the energy constraints in autonomous mobile agents (AMAs), such as unmanned vehicles, spiking neural networks (SNNs) are increasingly favored as a more efficient alternative to traditional artificial neural networks. AMAs employ multi-object detection (MOD) from multiple cameras to identify nearby objects while ensuring two essential objectives, (R1) timing guarantee and (R2) high accuracy for safety. In this paper, we propose RT-SNN, the first system design, aiming at achieving R1 and R2 in SNN-based MOD systems on AMAs. Leveraging the characteristic that SNNs gather feature data of input image termed as membrane potential, through iterative computation over multiple timesteps, RT-SNN provides multiple execution options with adjustable timesteps and a novel method for reusing membrane potential to support R1. Then, it captures how these execution strategies influence R2 by introducing a novel notion of mean absolute error and membrane confidence. Further, RT-SNN develops a new scheduling framework consisting of offline schedulability analysis for R1 and a run-time scheduling algorithm for R2 using the notion of membrane confidence. We deployed RT-SNN to Spiking-YOLO, the SNN-based MOD model derived from ANN-to-SNN conversion, and our experimental evaluation confirms its effectiveness in meeting the R1 and R2 requirements while providing significant energy efficiency.

Figures (5)

• Figure 1: Influence of (a) timestep variations and (b) membrane potential reuse on accuracy
• Figure 2: RT-SNN Methodology
• Figure 3: Precision on regression function $RF$ according to $g$
• Figure 4: Experiment results on various approaches
• Figure 5: Experiment results on RT-SNN with various parameters