Robust Bidirectional Associative Memory via Regularization Inspired by the Subspace Rotation Algorithm
Ci Lin, Tet Yeap, Iluju Kiringa, Biwei Zhang
TL;DR
The paper addresses the vulnerability of Bidirectional Associative Memory (BAM) to noise and adversarial perturbations when trained with gradient-based methods. It introduces Bidirectional Subspace Rotation Algorithm (B-SRA), a gradient-free training approach that optimizes weight matrices via subspace rotation, and identifies two core robustness principles: Orthogonal Weight Matrix (OWM) regularization and Gradient-Pattern Alignment (GPA). Building on this, the authors propose two regularizers for Bidirectional Backpropagation (B-BP) to mimic the robustness benefits of B-SRA, with the SAME configuration (joint OWM and GPA alignment) delivering the strongest resilience across attacks and memory capacities. Experimental results on MNIST and script datasets show that B-SRA outperforms B-BP under noise and a range of adversarial attacks, and that the SAME strategy sustains robustness in deeper, larger BAM architectures. These insights point to broader implications for designing resilient recurrent and attention-based models, suggesting extensions to Transformer and Hopfield-style networks and motivating targeted adversarial evaluations for BAM-like systems. $ rac{dE(t)}{dt} \le 0 $ indicates the energy-based stability during BAM inference, reinforcing the theoretical foundation of the proposed methods.
Abstract
Bidirectional Associative Memory (BAM) trained with Bidirectional Backpropagation (B-BP) often suffers from poor robustness and high sensitivity to noise and adversarial attacks. To address these issues, we propose a novel gradient-free training algorithm, the Bidirectional Subspace Rotation Algorithm (B-SRA), which significantly improves the robustness and convergence behavior of BAM. Through comprehensive experiments, we identify two key principles -- orthogonal weight matrices (OWM) and gradient-pattern alignment (GPA) -- as central to enhancing the robustness of BAM. Motivated by these findings, we introduce new regularization strategies into B-BP, resulting in models with greatly improved resistance to corruption and adversarial perturbations. We further conduct an ablation study across different training strategies to determine the most robust configuration and evaluate BAM's performance under a variety of attack scenarios and memory capacities, including 50, 100, and 200 associative pairs. Among all methods, the SAME configuration, which integrates both OWM and GPA, achieves the strongest resilience. Overall, our results demonstrate that B-SRA and the proposed regularization strategies lead to substantially more robust associative memories and open new directions for building resilient neural architectures.