Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributed Learning and Function Fusion in Reproducing Kernel Hilbert Space

Aneesh Raghavan, Karl Henrik Johansson

TL;DR

This work develops a forget-and-relearn distributed learning framework in Reproducing Kernel Hilbert Spaces for two agents and a fusion center. Each agent learns locally in its own RKHS with kernel $K^i$, uploads a function to a fusion center whose fused kernel is $K=K^1+K^2$, and then relearns the fused function via a regularized least-squares problem in the fused RKHS; the process uses uploading/downloading operators and a data-recovery fusion step to iteratively refine the global estimator. Theoretical results establish closed-form estimators at the agents, bounds and convergence properties of the learning operators to unity, and strong consistency with fixed-point guarantees; an example demonstrates the approach with heterogeneous features and a compact fused RKHS. The framework supports heterogeneous, privacy-preserving data fusion, and points to future directions in nonparametric extensions, transform-based fusion, and knowledge-transfer with reasoning integrations.

Abstract

We consider the problem of function estimation by a multi-agent system comprising of two agents and a fusion center. Each agent receives data comprising of samples of an independent variable (input) and the corresponding values of the dependent variable (output). The data remains local and is not shared with other members in the system. The objective of the system is to collaboratively estimate the function from the input to the output. To this end, we develop an iterative distributed algorithm for this function estimation problem. Each agent solves a local estimation problem in a Reproducing Kernel Hilbert Space (RKHS) and uploads the function to the fusion center. At the fusion center, the functions are fused by first estimating the data points that would have generated the uploaded functions and then subsequently solving a least squares estimation problem using the estimated data from both functions. The fused function is downloaded by the agents and is subsequently used for estimation at the next iteration along with incoming data. This procedure is executed sequentially and stopped when the difference between consecutively estimated functions becomes small enough. To analyze the algorithm, we define learning operators for the agents, fusion center and the system. We study the asymptotic properties of the norm of the learning operators and find sufficient conditions under which they converge to $1$. Given a sequence of data points, we define and prove the existence of the learning operator for the system. We prove that the porposed learning algorithm is consistent and demonstrate the same using an example. The paper has been submitted to L4DC 2024.

Distributed Learning and Function Fusion in Reproducing Kernel Hilbert Space

TL;DR

This work develops a forget-and-relearn distributed learning framework in Reproducing Kernel Hilbert Spaces for two agents and a fusion center. Each agent learns locally in its own RKHS with kernel , uploads a function to a fusion center whose fused kernel is , and then relearns the fused function via a regularized least-squares problem in the fused RKHS; the process uses uploading/downloading operators and a data-recovery fusion step to iteratively refine the global estimator. Theoretical results establish closed-form estimators at the agents, bounds and convergence properties of the learning operators to unity, and strong consistency with fixed-point guarantees; an example demonstrates the approach with heterogeneous features and a compact fused RKHS. The framework supports heterogeneous, privacy-preserving data fusion, and points to future directions in nonparametric extensions, transform-based fusion, and knowledge-transfer with reasoning integrations.

Abstract

We consider the problem of function estimation by a multi-agent system comprising of two agents and a fusion center. Each agent receives data comprising of samples of an independent variable (input) and the corresponding values of the dependent variable (output). The data remains local and is not shared with other members in the system. The objective of the system is to collaboratively estimate the function from the input to the output. To this end, we develop an iterative distributed algorithm for this function estimation problem. Each agent solves a local estimation problem in a Reproducing Kernel Hilbert Space (RKHS) and uploads the function to the fusion center. At the fusion center, the functions are fused by first estimating the data points that would have generated the uploaded functions and then subsequently solving a least squares estimation problem using the estimated data from both functions. The fused function is downloaded by the agents and is subsequently used for estimation at the next iteration along with incoming data. This procedure is executed sequentially and stopped when the difference between consecutively estimated functions becomes small enough. To analyze the algorithm, we define learning operators for the agents, fusion center and the system. We study the asymptotic properties of the norm of the learning operators and find sufficient conditions under which they converge to . Given a sequence of data points, we define and prove the existence of the learning operator for the system. We prove that the porposed learning algorithm is consistent and demonstrate the same using an example. The paper has been submitted to L4DC 2024.
Paper Structure (14 sections, 18 theorems, 33 equations, 4 figures, 1 algorithm)

This paper contains 14 sections, 18 theorems, 33 equations, 4 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. The Distributed Learning System
  3. The Learning Architecture
  4. Estimation at the agents
  5. Fusion Problem
  6. The Learning Algorithm
  7. Main Results
  8. Estimation at the Agents
  9. The Fusion Problem
  10. Consistency of the Learning Algorithm
  11. Example
  12. Conclusion and Future Work
  13. Appendix
  14. Fusion Space, Uploading and Downloading Operator

Key Result

Proposition 2

Let $\boldsymbol{\alpha^{1,*}_{n}} = ( \varrho^{i}_{n}\mathbf{K^{i}}+ \mathbf{\bar{K}^{i}}(x^{i}_n) \mathbf{\bar{K}^{i^T}}(x^{i}_n) )^{-1} ( \mathbf{\bar{K}^{i}}(x^{i}_n) y^{i}_{n} + \varrho^{i}_{n}\mathbf{K^{i}} \boldsymbol{\bar{\alpha}^{1}_{n-1}})$. Then, $f^{i,*}_{n} = \boldsymbol{\alpha^{{1,*}

Figures (4)

  • Figure 1: The Learning Architecture
  • Figure 2: The learning operator at stage $n$, $\mathbb{T}_{n}$, obtained through composition of operators, $\bar{T}, T, \hat{T}.$
  • Figure 3: True function and estimated functions at: (right) Agent 1 and (left) Agent 2.
  • Figure 4: True function and estimated functions at Fusion Center.

Theorems & Definitions (26)

  • Definition 1
  • Proposition 2
  • Definition 3
  • Proposition 4
  • Proposition 5
  • Definition 6
  • Proposition 7
  • Definition 8
  • Proposition 9
  • Definition 10
  • ...and 16 more