Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Sinkhorn

Jeremy Kulcsar, Vyacheslav Kungurtsev, Georgios Korpas, Giulio Giaconi, William Shoosmith

TL;DR

This work extends entropy-regularized discrete OT by introducing Federated Sinkhorn, enabling distributed computation of the OT plan $P=\text{diag}(u)K\text{diag}(v)$ across multiple clients while preserving data privacy. It develops four variants across synchronous/asynchronous and all-to-all/star topologies, with convergence guarantees for the centralized-like limits under standard assumptions and damping, respectively. Empirical results on synthetic data and a financial risk example show consistent convergence and reveal a nuanced trade-off between computation and communication, with speedups being limited under typical hardware but potential gains for very large-scale problems. The framework enables privacy-preserving collaboration in domains like finance, where data distribution and regulatory constraints hinder centralized OT solutions, and connects to robust risk assessment via the Blanchet–Murphy Wasserstein formulation. The work thus contributes theoretically and empirically to scalable, private OT computation in federated environments, with practical implications for risk modeling and distributed analytics.

Abstract

In this work we investigate the potential of solving the discrete Optimal Transport (OT) problem with entropy regularization in a federated learning setting. Recall that the celebrated Sinkhorn algorithm transforms the classical OT linear program into strongly convex constrained optimization, facilitating first order methods for otherwise intractably large problems. A common contemporary setting that remains an open problem as far as the application of Sinkhorn is the presence of data spread across clients with distributed inter-communication, either due to clients whose privacy is a concern, or simply by necessity of processing and memory hardware limitations. In this work we investigate various natural procedures, which we refer to as Federated Sinkhorn, that handle distributed environments where data is partitioned across multiple clients. We formulate the problem as minimizing the transport cost with an entropy regularization term, subject to marginal constraints, where block components of the source and target distribution vectors are locally known to clients corresponding to each block. We consider both synchronous and asynchronous variants as well as all-to-all and server-client communication topology protocols. Each procedure allows clients to compute local operations on their data partition while periodically exchanging information with others. We provide theoretical guarantees on convergence for the different variants under different possible conditions. We empirically demonstrate the algorithms performance on synthetic datasets and a real-world financial risk assessment application. The investigation highlights the subtle tradeoffs associated with computation and communication time in different settings and how they depend on problem size and sparsity.

Federated Sinkhorn

TL;DR

This work extends entropy-regularized discrete OT by introducing Federated Sinkhorn, enabling distributed computation of the OT plan across multiple clients while preserving data privacy. It develops four variants across synchronous/asynchronous and all-to-all/star topologies, with convergence guarantees for the centralized-like limits under standard assumptions and damping, respectively. Empirical results on synthetic data and a financial risk example show consistent convergence and reveal a nuanced trade-off between computation and communication, with speedups being limited under typical hardware but potential gains for very large-scale problems. The framework enables privacy-preserving collaboration in domains like finance, where data distribution and regulatory constraints hinder centralized OT solutions, and connects to robust risk assessment via the Blanchet–Murphy Wasserstein formulation. The work thus contributes theoretically and empirically to scalable, private OT computation in federated environments, with practical implications for risk modeling and distributed analytics.

Abstract

In this work we investigate the potential of solving the discrete Optimal Transport (OT) problem with entropy regularization in a federated learning setting. Recall that the celebrated Sinkhorn algorithm transforms the classical OT linear program into strongly convex constrained optimization, facilitating first order methods for otherwise intractably large problems. A common contemporary setting that remains an open problem as far as the application of Sinkhorn is the presence of data spread across clients with distributed inter-communication, either due to clients whose privacy is a concern, or simply by necessity of processing and memory hardware limitations. In this work we investigate various natural procedures, which we refer to as Federated Sinkhorn, that handle distributed environments where data is partitioned across multiple clients. We formulate the problem as minimizing the transport cost with an entropy regularization term, subject to marginal constraints, where block components of the source and target distribution vectors are locally known to clients corresponding to each block. We consider both synchronous and asynchronous variants as well as all-to-all and server-client communication topology protocols. Each procedure allows clients to compute local operations on their data partition while periodically exchanging information with others. We provide theoretical guarantees on convergence for the different variants under different possible conditions. We empirically demonstrate the algorithms performance on synthetic datasets and a real-world financial risk assessment application. The investigation highlights the subtle tradeoffs associated with computation and communication time in different settings and how they depend on problem size and sparsity.

Paper Structure

This paper contains 45 sections, 2 theorems, 39 equations, 28 figures, 36 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contributions
  4. Algorithms
  5. Privacy Regime 1 - "All-to-All" federation
  6. Synchronous Federated Sinkhorn All-to-All
  7. Asynchronous Federated Sinkhorn All-to-All
  8. Privacy Regime 2 - "Star"-Network Federation
  9. Convergence Properties
  10. Experiments on Synthetic Datasets
  11. Study of the regularisation term $\epsilon$
  12. Properties of Distributed Computation
  13. Environment
  14. Details on the synchronous federation
  15. Coherence check
  16. ...and 30 more sections

Key Result

Proposition 1

Let $c\in \mathbb{N}_+$ be the number of clients, each holding local positive probability vectors $a^{(i)}, b^{(i)} \in$$\mathbb{R}_{+}^m$, such that $a=\left[a^{(1)} ; \ldots ; a^{(c)}\right] \in \mathbb{R}_{+}^n$ and $b=\left[b^{(1)} ; \ldots ; b^{(c)}\right] \in \mathbb{R}_{+}^n$, where $n=c m$.

Figures (28)

  • Figure 1: Illustration of how $a$, $b$ and $K$ are sliced in the case of $n=4$ and $c=2$ (which implies $m=2$).
  • Figure 2: Illustration of a full step in the iterative All-to-All process. Each node $i$ receives slices of $u$ from other nodes, concatenates them into a global $u$, uses its local $K^T_i$ and $b_i$ to compute $r_i=K^T_i u$ and $v_i=b_i / r_i$, and sends slice $v_i$ to other nodes. Then, each node does the same respective operations with $v$, $K_i$ and $a_i$.
  • Figure 3: Illustration of a step in the iterative Star-Network process. The clients receive slices of $r$ from the previous iteration step, use them to compute the $u_j$ and send them to the server node. The server then uses the global $K$ to compute $q$ and scatter it to the nodes, so they can compute the slices $v_j$, and so on.
  • Figure 4: Marginal error and objective function value against number of iterations. The smaller epsilon is, the higher the number of iterations to reach convergence. Note that the marginal errors on $a$ and $b$ differ due to the algorithm starting with $u$.
  • Figure 5: Logarithmic scale is used to visualise the behaviour of the function.
  • ...and 23 more figures

Theorems & Definitions (4)

  • Remark 1
  • Proposition 1: Synchronous Federated Sinkhorn convergence
  • Proposition 2: Asynchronous Federated Sinkhorn all-to-all algorithm convergence
  • proof