Biologically Plausible Training of Deep Neural Networks Using a Top-down Credit Assignment Network

TL;DR

A brain-inspired credit diffusion mechanism is introduced, significantly reducing the TDCA-network's parameter complexity, thereby greatly accelerating training without compromising the network's performance, and results indicate that the TDCA-network holds promising potential to train neural networks across diverse architectures.

Abstract

Despite the widespread adoption of Backpropagation algorithm-based Deep Neural Networks, the biological infeasibility of the BP algorithm could potentially limit the evolution of new DNN models. To find a biologically plausible algorithm to replace BP, we focus on the top-down mechanism inherent in the biological brain. Although top-down connections in the biological brain play crucial roles in high-level cognitive functions, their application to neural network learning remains unclear. This study proposes a two-level training framework designed to train a bottom-up network using a Top-Down Credit Assignment Network (TDCA-network). The TDCA-network serves as a substitute for the conventional loss function and the back-propagation algorithm, widely used in neural network training. We further introduce a brain-inspired credit diffusion mechanism, significantly reducing the TDCA-network's parameter complexity, thereby greatly accelerating training without compromising the network's performance.Our experiments involving non-convex function optimization, supervised learning, and reinforcement learning reveal that a well-trained TDCA-network outperforms back-propagation across various settings. The visualization of the update trajectories in the loss landscape indicates the TDCA-network's ability to bypass local minima where BP-based trajectories typically become trapped. The TDCA-network also excels in multi-task optimization, demonstrating robust generalizability across different datasets in supervised learning and unseen task settings in reinforcement learning. Moreover, the results indicate that the TDCA-network holds promising potential to train neural networks across diverse architectures.

Figures (7)

• Figure 1: The schematic diagram of our novel top-down learning framework.
• Figure 2: Top-Down Learning Framework for Optimization: BP-Free Navigation in Parameter Spaces
• Figure 3: Multi Reinforcement Task Optimization in Top-Down Learning: Framework and Environments.
• Figure 4: Top-Down Credit Assignment Learning Framework for Classification Tasks in Supervised Learning: Dense and Sparse Credit Distribution.
• Figure S1: Convergence of the PGPE algorithm during TDCA-network training. The first 500 iterations of the TDCA network training's outer loop are selected for analysis. Convergence curves of these two metrics across various data sets are depicted in the figures. Panels (a) and (b) illustrate the convergence process of cross-entropy and accuracy, respectively.