A stochastic preconditioned Douglas-Rachford splitting method for saddle-point problems

TL;DR

This work develops a stochastic, relaxed, preconditioned Douglas–Rachford splitting method (SRPDR) for solving convex–concave saddle-point problems with separable dual variables. It establishes almost-sure convergence of the stochastic iterates to a saddle point and derives an ergodic convergence rate of $\mathcal{O}(1/K)$ for the restricted primal–dual gap and primal errors, under standard stochastic quasi-Fejér monotonicity and Opial’s lemma arguments. The framework extends to a quadratic setting (SRPDRQ) with a more compact update and retains similar convergence properties. Numerical experiments on TGV–KL deblurring and LIBSVM-style binary classification demonstrate faster, more stable convergence compared with SPDHG and PDHG, validating the practical impact of preconditioning and over-relaxation in large-scale, nonsmooth saddle-point problems.

Abstract

In this article, we propose and study a stochastic and relaxed preconditioned Douglas--Rachford splitting method to solve saddle-point problems that have separable dual variables. We prove the almost sure convergence of the iteration sequences in Hilbert spaces for a class of convex-concave and nonsmooth saddle-point problems. We also provide the sublinear convergence rate for the ergodic sequence concerning the expectation of the restricted primal-dual gap functions. Numerical experiments show the high efficiency of the proposed stochastic and relaxed preconditioned Douglas--Rachford splitting methods.

Figures (5)

• Figure 1: TGV-KL deblurring problem. $\emph{(a)}$, $\emph{(}b)$ are normalized by $d_1\times d_2$ and in a double-logarithmic scale. Image $\emph{(}b)$ shows the speed comparison measured in terms of primal error, which is given by $( \mathcal{P}(x^{K})-\mathcal{P}(x^*) )/ \mathcal{P}(x^*)$.
• Figure 2: Examples of variational TGV-KL deblurring after 300 epochs with uniform sampling.
• Figure 3: Convergence rate of relative primal error concerning epochs for dataset Gisette and Madelon. The regularization parameter is set to be $1 \times 10^{-6}$.
• Figure 4: Convergence rate of relative primal error concerning epochs for dataset Gisette and Madelon. The regularization parameter is set to be $1 \times 10^{-4}$.
• Figure 5: Comparison between SRPDR and SPRDRQ on primal errors with respect to the epochs.

Theorems & Definitions (35)

• Lemma 1: BS
• Proposition 1: BS
• Proposition 2
• Lemma 2
• Remark 1
• Proposition 3
• Lemma 3
• Theorem 1
• Lemma 4
• proof