Optimising Graph Representation for Hardware Implementation of Graph Convolutional Networks for Event-based Vision

TL;DR

This work tackles real-time object detection on event-based vision data by designing a hardware-friendly graph generation module for FPGAs. It introduces graph representation optimisations—time-directed edges, normalization, a neighbour matrix, and unique-event handling—to reduce edge counts while preserving detection accuracy. The hardware architecture, implemented in SystemVerilog on a Xilinx ZCU104 at 250 MHz, achieves near real-time throughput (about 9.6 events per μs) with only a small mAP loss (0.08% on N-Caltech101) using SIZE=256 and R=3. The results demonstrate a viable path to embedded, energy-efficient event-based GCN processing and point to future work on streaming processing and data filtering.

Abstract

Event-based vision is an emerging research field involving processing data generated by Dynamic Vision Sensors (neuromorphic cameras). One of the latest proposals in this area are Graph Convolutional Networks (GCNs), which allow to process events in its original sparse form while maintaining high detection and classification performance. In this paper, we present the hardware implementation of a~graph generation process from an event camera data stream, taking into account both the advantages and limitations of FPGAs. We propose various ways to simplify the graph representation and use scaling and quantisation of values. We consider both undirected and directed graphs that enable the use of PointNet convolution. The results obtained show that by appropriately modifying the graph representation, it is possible to create a~hardware module for graph generation. Moreover, the proposed modifications have no significant impact on object detection performance, only 0.08% mAP less for the base model and the N-Caltech data set.Finally, we describe the proposed hardware architecture of the graph generation module.

Figures (1)

• Figure 1: The proposed hardware module scheme