Backpropagation-Free 4D Continuous Ant-Based Neural Topology Search

TL;DR

BP-Free CANTS introduces a backpropagation-free NAS method that operates in a 4D continuous search space, with the fourth dimension encoding candidate synaptic weights. It uses a distributed asynchronous framework of cant agents guided by pheromones, DBSCAN-based path condensation, and communal weight sharing to jointly search for neural topology and weights without gradient-based optimization. Empirical results on coal-plant time-series data show competitive performance relative to ANTS and BP-CANTS while achieving substantial reductions in computational time. This approach broadens NAS applicability to unbounded architectures and non-differentiable components, with potential extensions to other network types and continuous multi-dimensional search spaces.

Abstract

Continuous Ant-based Topology Search (CANTS) is a previously introduced novel nature-inspired neural architecture search (NAS) algorithm that is based on ant colony optimization (ACO). CANTS utilizes a continuous search space to indirectly-encode a neural architecture search space. Synthetic ant agents explore CANTS' continuous search space based on the density and distribution of pheromones, strongly inspired by how ants move in the real world. This continuous search space allows CANTS to automate the design of artificial neural networks (ANNs) of any size, removing a key limitation inherent to many current NAS algorithms that must operate within structures of a size that is predetermined by the user. This work expands CANTS by adding a fourth dimension to its search space representing potential neural synaptic weights. Adding this extra dimension allows CANTS agents to optimize both the architecture as well as the weights of an ANN without applying backpropagation (BP), which leads to a significant reduction in the time consumed in the optimization process: at least an average of 96% less time consumption with very competitive optimization performance, if not better. The experiments of this study - using real-world data - demonstrate that the BP-Free CANTS algorithm exhibits highly competitive performance compared to both CANTS and ANTS while requiring significantly less operation time.

Figures (8)

• Figure 1: The BP-free CANTS schematic graph.
• Figure 2: The CANTS asynchronous design.
• Figure 3: CANTS paths & Architecture Building
• Figure 4: CANTS paths & Architecture Building (continued):
• Figure 5: Potential cant speed pattern based on its $y$ position, $r_1$, and $r_2$ values.