Cross-Layer Design of Vector-Symbolic Computing: Bridging Cognition and Brain-Inspired Hardware Acceleration

TL;DR

The paper addresses the fragmentation between VSA theory and hardware realization by articulating a cross-layer hardware/software co-design framework for Vector Symbolic Architectures. It surveys HD-vector foundations, learning paradigms, and a spectrum of digital and memory-centric hardware approaches, then introduces kernel-centric design and cross-layer features to bridge software and hardware. A concrete in-memory computing template demonstrates hierarchical cognition through a multi-modal perception case study, highlighting gains in efficiency, flexibility, and scalability. The work also outlines open challenges in modular kernel formulation, hardware mapping tradeoffs, benchmarking, and NVSA integration, emphasizing a push toward cohesive, reconfigurable cognitive hardware platforms.

Abstract

Vector Symbolic Architectures (VSAs) have been widely deployed in various cognitive applications due to their simple and efficient operations. The widespread adoption of VSAs has, in turn, spurred the development of numerous hardware solutions aimed at optimizing their performance. Despite these advancements, a comprehensive and unified discourse on the convergence of hardware and algorithms in the context of VSAs remains somewhat limited. The paper aims to bridge the gap between theoretical software-level explorations and the development of efficient hardware architectures and emerging technology fabrics for VSAs, providing insights from the co-design aspect for researchers from either side. First, we introduce the principles of vector-symbolic computing, including its core mathematical operations and learning paradigms. Second, we provide an in-depth discussion on hardware technologies for VSAs, analyzing analog, mixed-signal, and digital circuit design styles. We compare hardware implementations of VSAs by carrying out detailed analysis of their performance characteristics and tradeoffs, allowing us to extract design guidelines for the development of arbitrary VSA formulations. Third, we discuss a methodology for cross-layer design of VSAs that identifies synergies across layers and explores key ingredients for hardware/software co-design of VSAs. Finally, as a concrete demonstration of this methodology, we propose the first in-memory computing hierarchical cognition hardware system, showcasing the efficiency, flexibility, and scalability of this co-design approach. The paper concludes with a discussion of open research challenges for future explorations.

Figures (9)

• Figure 1: An overview of VSA system design methodologies, including (a) an alternative hardware/software co-design framework; (b) a demonstration through a co-design case study.
• Figure 2: Background of Vector-Symbolic Computing. (a) Orthogonality in high dimensions becomes more pronounced, i.e., density concentration becomes sharper around 0.5, as vector dimensionality increases. (b) Basic elements of vector-symbolic computations: item memory and algebraic operations. (c) A superposed, higher-order scene vector constructed via vector-symbolic operations.
• Figure 3: An example of an arbitrary VSA classification application with its three operating modes: encoding, tree search, and associative search.
• Figure 4: Visualization of kernel formulations for the example in Figure \ref{['fig:kernels-ex']}: (a) spatial kernel formulation; (b) temporal kernel formulation.
• Figure 5: Mapping of VSA compute platforms to application domains based on latency/throughput and energy constraints.