Sign-Symmetry Learning Rules are Robust Fine-Tuners

TL;DR

The paper addresses the vulnerability of standard backpropagation to gradient-based adversarial attacks by proposing a hybrid training regime: pretrain with backpropagation and then fine-tune with Bio-plausible Sign-Symmetry learning rules. By evaluating across image classification and hashing-based image retrieval, using backbones like AlexNet, VGG-16, and ResNet-18, the study demonstrates that Sign-Symmetry fine-tuning can match or even surpass BP in accuracy while yielding substantially improved adversarial robustness. The key contributions include a systematic comparison of Sign-Symmetry variants (uSF, frSF, brSF), evidence of robustness gains against white-box and hashing attacks, and analysis showing black-box attacks do not negate the benefits. This work suggests a practical pathway to integrate biologically inspired learning with standard BP pipelines to enhance reliability without sacrificing performance, with broad implications for robust deep learning systems.

Abstract

Backpropagation (BP) has long been the predominant method for training neural networks due to its effectiveness. However, numerous alternative approaches, broadly categorized under feedback alignment, have been proposed, many of which are motivated by the search for biologically plausible learning mechanisms. Despite their theoretical appeal, these methods have consistently underperformed compared to BP, leading to a decline in research interest. In this work, we revisit the role of such methods and explore how they can be integrated into standard neural network training pipelines. Specifically, we propose fine-tuning BP-pre-trained models using Sign-Symmetry learning rules and demonstrate that this approach not only maintains performance parity with BP but also enhances robustness. Through extensive experiments across multiple tasks and benchmarks, we establish the validity of our approach. Our findings introduce a novel perspective on neural network training and open new research directions for leveraging biologically inspired learning rules in deep learning.

Figures (6)

• Figure 1: Credit assignment methods can be ranked based on the amount of weight transport each method employs. Backpropagation is known to use full-weight transport, as the algorithm utilizes the same forward weight matrix to conduct the backward pass. Uniform Sign-concordant Feedback (uSF) uses the sign of the weight matrix as a backward matrix, thus using less information than BP. Further Fixed Random Magnitude Sign-Concordant Feedback (frSF) and Batchwise Random Magnitude Sign-concordant Feedback (brSF) add approximation to the sign of the weight matrix, which introduces more randomness and uses less weight information. Finally, Feedback Alignment (FA) uses no weight transport, relying instead on a fixed random matrix to conduct the backward pass.
• Figure 2: Accuracy of ResNet18 under FGSM (top row) and PGD (bottom row) adversarial attacks. Note the gradual decline in Sign-Symmetry methods compared to BP's sharp drop.
• Figure 3: Adversarial robustness of VGG-16 under HAG and SDHA attacks. Trends mirror AlexNet: BP performance collapses sharply (e.g., mAP@5000 drops by 28.65% on CIFAR-10 at $\epsilon=0.5$), while Sign-Symmetry methods retain robustness across all datasets. Metrics: mAP@5000 for CIFAR-10/MSCOCO, mAP@1000 for ImageNet.
• Figure 4: Accuracy of AlexNet under FGSM (top row) and PGD (bottom row) adversarial attacks across different datasets. Higher $\epsilon$ values indicate stronger perturbations.
• Figure 5: Accuracy of VGG16 under FGSM (top row) and PGD (bottom row) adversarial attacks across different datasets. The robustness gap between methods is particularly pronounced at larger $\epsilon$ values.