Multi-Neuron Representations of Hierarchical Concepts in Spiking Neural Networks
Nancy A. Lynch
TL;DR
This work studies how to represent hierarchical concepts in layered spiking neural networks in a fault-tolerant manner by using multiple input representatives per concept across three architectures: feed-forward with high connectivity, feed-forward with low connectivity, and layered networks with lateral edges. The core method combines formal concept-hierarchy embeddings with redundancy and probabilistic analysis (Chernoff bounds and union bounds) to quantify the probability of correct recognition under partial information and initial neuron failures, showing that recognition improves with more representatives and degrades with higher failure probability. It provides main theorems guaranteeing recognition under specified separation conditions (e.g., $r_1 \le r_2 p (1-\zeta)$) and extends to low connectivity and lateral-edge networks via refined connectivity (including the Class Assumption) and corresponding survival lemmas. The paper also outlines learning strategies inspired by LM21 and assembly calculus to construct these representations, and discusses limitations, potential extensions, and directions for simulation-based validation. These results contribute a formal, fault-tolerant framework for hierarchical concept representations that could inform brain-inspired AI and cognitive modeling of structured knowledge in SNNs.
Abstract
We describe how hierarchical concepts can be represented in three types of layered neural networks. The aim is to support recognition of the concepts when partial information about the concepts is presented, and also when some of the neurons in the network might fail. Our failure model involves initial random failures. The three types of networks are: feed-forward networks with high connectivity, feed-forward networks with low connectivity, and layered networks with low connectivity and with both forward edges and "lateral" edges within layers. In order to achieve fault-tolerance, the representations all use multiple representative neurons for each concept. We show how recognition can work in all three of these settings, and quantify how the probability of correct recognition depends on several parameters, including the number of representatives and the neuron failure probability. We also discuss how these representations might be learned, in all three types of networks. For the feed-forward networks, the learning algorithms are similar to ones used in [4], whereas for networks with lateral edges, the algorithms are generally inspired by work on the assembly calculus [3, 6, 7].