AtPatch: Debugging Transformers via Hot-Fixing Over-Attention
Shihao Weng, Yang Feng, Jincheng Li, Yining Yin, Xiaofei Xie, Jia Liu
TL;DR
AtPatch targets backdoor and fairness vulnerabilities in transformer models by addressing the over-attention phenomenon through runtime, attention-map hot-patching. It builds a Detector via offline delta-debugging-inspired contrastive learning to identify anomalous attention columns and applies selective, parameter-free patches that replace anomalous columns with benign references while rebalancing the rest, preserving performance on clean data. Across six benchmarks and architectures, AtPatch achieves near-perfect backdoor mitigation and fairness improvements with minimal accuracy loss, outperforming four baselines in both effectiveness and efficiency, and maintaining low online overhead. The approach is architecture- and data-agnostic, enabling deployment on existing models and offering a practical, scalable direction for real-world transformer debugging and safety.
Abstract
Transformer-based deep neural networks (DNNs) affected by backdoor attacks and unfairness typically exhibit anomalous attention patterns, leading to over-attend to backdoor triggers or protected attributes. Existing neuron-editing mitigation strategies often struggle to handle such situation and most of them lack flexibility and tend to distort feature representations. Motivated by such over-attention phenomenon and software engineering paradigms such as delta debugging and hot patching, we propose AtPatch, a hot-fix method that dynamically redistributes attention maps during model inference. Specifically, for a given input, AtPatch first extracts the attention map from the model's inference process. Then, it uses a pre-trained detector to identify anomalous columns and replace them with unified benign attention. Then, AtPatch rescales other columns to mitigate the impact of over-attention. Finally, AtPatch returns the redistributed attention map to the model for continued inference. Notably, if the detector does not report any anomalous columns, AtPatch directly returns the original attention map to the model. Unlike existing techniques, AtPatch selectively redistributes the attention map, making it better at preserving the model's original functionality. Furthermore, AtPatch's on-the-fly nature allows it to work without modifying model parameters or retraining, making it better suited for deployed models. We conducted extensive experiments to validate AtPatch. Experimental results show that, compared to existing methods, AtPatch can more effectively mitigate backdoor attacks and unfairness while better preserving the model's original functionality.