Bant: Byzantine Antidote via Trial Function and Trust Scores
Gleb Molodtsov, Daniil Medyakov, Sergey Skorik, Nikolas Khachaturov, Shahane Tigranyan, Vladimir Aletov, Aram Avetisyan, Martin Takáč, Aleksandr Beznosikov
TL;DR
This work tackles Byzantine attacks in distributed and federated learning by combining a server-held trial function with trust-based weighting to filter malicious updates, requiring only a single honest worker and data similarity.Two main methods are introduced: Bant, which uses trial-function reductions to weight client gradients with momentum, and AutoBant, which optimizes client weights via Mirror Descent to relax trust assumptions and handle nonconvex objectives.The authors extend these methods to practical settings (Local SGD, partial participation, adaptive optimizers) and provide convergence guarantees under standard smoothness/convexity assumptions, with an explicit error term from trial-function approximation. Extensive experiments on CIFAR-10, ECG data, and Learning-to-Rank demonstrate robust performance against multiple Byzantine attacks, outperforming several prior defenses especially when honest devices are not in the majority.Overall, Bant, AutoBant, and SimBant offer a versatile, theoretically grounded framework for Byzantine-robust distributed optimization with broad applicability and practical relevance.
Abstract
Recent advancements in machine learning have improved performance while also increasing computational demands. While federated and distributed setups address these issues, their structures remain vulnerable to malicious influences. In this paper, we address a specific threat: Byzantine attacks, wherein compromised clients inject adversarial updates to derail global convergence. We combine the concept of trust scores with trial function methodology to dynamically filter outliers. Our methods address the critical limitations of previous approaches, allowing operation even when Byzantine nodes are in the majority. Moreover, our algorithms adapt to widely used scaled methods such as Adam and RMSProp, as well as practical scenarios, including local training and partial participation. We validate the robustness of our methods by conducting extensive experiments on both public datasets and private ECG data collected from medical institutions. Furthermore, we provide a broad theoretical analysis of our algorithms and their extensions to the aforementioned practical setups. The convergence guaranties of our methods are comparable to those of classical algorithms developed without Byzantine interference.