An Integrated Toolbox for Creating Neuromorphic Edge Applications

TL;DR

This paper tackles the challenge of deploying neuromorphic spiking neural networks (SNNs) at the edge by introducing CARLsim++, an integrated toolbox that combines a graphical user interface, robust input/output interfaces, and a modular plugin architecture. It extends the CARLsim SNN framework with YARP-based robotics I/O, a reproducible YCM-based build system, and a neurorobotics-focused E-Puck demonstration, enabling rapid development and deployment of neuromorphic edge applications. Key contributions include GUI-based model creation, real-time I/O integration, cloud-native deployment support via Docker, and DevOps tooling (logging, monitoring, robot test framework) to streamline edge workflows. The results demonstrate end-to-end workflows from simulation to physical robot control and digital twins, highlighting the framework’s potential to bridge neuroscience research with practical, field-ready neuromorphic systems.

Abstract

Spiking Neural Networks (SNNs) and neuromorphic models are more efficient and have more biological realism than the activation functions typically used in deep neural networks, transformer models and generative AI. SNNs have local learning rules, are able to learn on small data sets, and can adapt through neuromodulation. Although research has shown their advantages, there are still few compelling practical applications, especially at the edge where sensors and actuators need to be processed in a timely fashion. One reason for this might be that SNNs are much more challenging to understand, build, and operate due to their intrinsic properties. For instance, the mathematical foundation involves differential equations rather than basic activation functions. To address these challenges, we have developed CARLsim++. It is an integrated toolbox that enables fast and easy creation of neuromorphic applications. It encapsulates the mathematical intrinsics and low-level C++ programming by providing a graphical user interface for users who do not have a background in software engineering but still want to create neuromorphic models. Developers can easily configure inputs and outputs to devices and robots. These can be accurately simulated before deploying on physical devices. CARLsim++ can lead to rapid development of neuromorphic applications for simulation or edge processing.

Figures (5)

• Figure 1: CARLsim++ GUI and I/O Interface. Top: Screenshot showing the neural activity of the SNN model in the GUI. Left: Webots simulation of E-Puck robot controlled by CARLsim I/O interface. Center: I/O interface shows listing channels for the E-Puck's sensors and actuators. Right: Porting model to physical E-Puck robot as it detects a wall by its front-right proximity sensor, and performs obstacle avoidance.
• Figure 2: The E-Puck robot. (a) E-Puck facing a wall at the front right. (b) E-Puck monitor application showing the corresponding sensor values. Note the high value of proximity sensor IR1. (c) Location of the proximity sensors. Reproduced from the E-Puck online documentation.
• Figure 3: CARLsim's GUI provides plug-ins in a drop-down menu list for adding neuron groups with sensory and motor neurons. (a) Dialog of the plug-in to add neuron groups mapped to E-Puck's proximity sensors and motor actuators. (b) 3D Visualization and selection of sensory and motor neuron groups.
• Figure 4: Connecting sensory and motor neurons. (a) A minimalistic neuromorphic brain that controls the E-Puck robot. Its layout was inspired by the topography of the motor and sensory cortex. The blue markers indicate similar positions of the sensory neurons to the proximity sensors shown in Fig. \ref{['subfig:EPuckSensors']}. The motor neurons are located close to the sensory neurons and at the corresponding side. (b) Schematic of sensory neurons connecting to the motor neurons. If an obstacle comes in front of E-Puck, the ToF sensory neurons excite the backward motor neurons in VELleft and VELright and E-Puck slows down, respectively stops. Otherwise, the population-rate coding of the two sensory neurons in the neuron group ToF (ctx.tof) are configured in such way that they drive E-Puck forward. Fig. \ref{['fig:io']} shows E-Puck detecting a wall by its front-right proximity sensor Prox1 and the firing of the corresponding sensory neuron ps[1] in the neuron group PS (ctx.ps). The exemplary proximity sensor neuron ps[1] excites the proximal forward motor neuron of VELright and the distal backward motor neuron of VELleft making E-Puck turning left and therefore away from the wall. All of the eight proximity sensory neurons are connected in such a way. This enabled E-Puck to explore a T-shape maze without getting stuck in corners or passages.
• Figure 5: Experiment showing E-Puck exploring an area restricted by walls and avoiding collisions. (a) Start position of E-Puck. (b) Trajectory showing E-Puck exploring the field and changing course when coming near to a wall. Analysis of neural activity and interactive control of the monitoring. (c) The CARLsim++ plug-in for CARLsim OAT, which allows to start and stop an OAT monitor and to interactively access its results. (d) Spike trains of neuron groups for sensors and actuators. The plots were generated by OAT and show the neuron group names as they were configured in CARLsim GUI. The namespace ctx indicates that the groups belong to the motor and sensory cortex. The time for the plot was arbitrary chosen to indicate high neural activity. At about 2.5s, several proximity sensor neurons were spiking, indicating that E-puck was near or in a corner. Consequently, ToF was detecting obstacles and the motor neurons encoded no forward movement. At about 2.8s, the silence of the ctx.ps neurons and the lower ctx.tof and ctx.vel neurons indicate that E-Puck was moving straight ahead.