Application-oriented automatic hyperparameter optimization for spiking neural network prototyping
Vittorio Fra
TL;DR
This work tackles the challenge of hyperparameter optimization for spiking neural networks by proposing an application-oriented automatic HPO pipeline built on the NNI toolkit. It details a modular methodology with four Python components that streamline experiment configuration, optimization control, data preparation, and model training, and demonstrates the approach on an e-prop-based Braille reading use case. The results illustrate the pipeline’s ability to identify high-performing hyperparameter configurations, achieving strong test performance on the Braille dataset, with model selection driven by validation accuracy. Additionally, the paper surveys published works using this pipeline, underscoring its adaptability to diverse neuromorphic tasks and its potential to accelerate SNN prototyping and deployment.
Abstract
Hyperparameter optimization (HPO) is of paramount importance in the development of high-performance, specialized artificial intelligence (AI) models, ranging from well-established machine learning (ML) solutions to the deep learning (DL) domain and the field of spiking neural networks (SNNs). The latter introduce further complexity due to the neuronal computational units and their additional hyperparameters, whose inadequate setting can dramatically impact the final model performance. At the cost of possible reduced generalization capabilities, the most suitable strategy to fully disclose the power of SNNs is to adopt an application-oriented approach and perform extensive HPO experiments. To facilitate these operations, automatic pipelines are fundamental, and their configuration is crucial. In this document, the Neural Network Intelligence (NNI) toolkit is used as reference framework to present one such solution, with a use case example providing evidence of the corresponding results. In addition, a summary of published works employing the presented pipeline is reported as a potential source of insights into application-oriented HPO experiments for SNN prototyping.