Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence

TL;DR

This paper essentially shows that, under reasonable parameter settings, a distributed system can not only withstand noisier zeroth-order agents but can even benefit from integrating such agents into the optimization process, rather than ignoring their information.

Abstract

Distributed optimization is the standard way of speeding up machine learning training, and most of the research in the area focuses on distributed first-order, gradient-based methods. Yet, there are settings where some computationally-bounded nodes may not be able to implement first-order, gradient-based optimization, while they could still contribute to joint optimization tasks. In this paper, we initiate the study of hybrid decentralized optimization, studying settings where nodes with zeroth-order and first-order optimization capabilities co-exist in a distributed system, and attempt to jointly solve an optimization task over some data distribution. We essentially show that, under reasonable parameter settings, such a system can not only withstand noisier zeroth-order agents but can even benefit from integrating such agents into the optimization process, rather than ignoring their information. At the core of our approach is a new analysis of distributed optimization with noisy and possibly-biased gradient estimators, which may be of independent interest. Our results hold for both convex and non-convex objectives. Experimental results on standard optimization tasks confirm our analysis, showing that hybrid first-zeroth order optimization can be practical, even when training deep neural networks.

Figures (7)

• Figure 1: Number of random vectors (rv) impact on the biased/unbiased ZO estimators Accuracy (Acc), using a CNN model on MNIST.
• Figure 2: Validation loss vs. various population configurations for regression model on MNIST.
• Figure 3: Validation accuracy (Acc) comparison between the hybrid and mono-type estimator population for ResNet-18 on the CIFAR-10 dataset.
• Figure 4: Validation loss comparison between the hybrid and mono-type estimator population for transformer model on the synthetic Brackets dataset.
• Figure 5: Impact of the learning rate (lr) on the validation loss, using a regression model on MNIST with 3 FO and 90 ZO nodes.

Theorems & Definitions (50)

• Definition 1
• Definition 2: Zeroth-order estimator
• Lemma 1: nesterov2017random, Theorem 1.1 in balasubramanian2019zeroth
• Theorem 1
• Lemma 2
• Lemma 3
• Lemma 4
• Lemma 5
• proof
• Definition 3: Gamma