Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Collaborative likelihood-ratio estimation over graphs

Alejandro de la Concha, Nicolas Vayatis, Argyris Kalogeratos

TL;DR

The paper extends likelihood-ratio estimation to graphs by introducing GRULSIF, a non-parametric, graph-regularized method that learns node-specific relative likelihood-ratios $r_v^{\alpha}$ in a shared VV-RKHS. It combines a nodewise χ^2-based loss with a Laplacian penalty to exploit graph smoothness, and leverages a Representer theorem to yield a tractable finite-dimensional optimization solved via Cyclic Block Coordinate Gradient Descent, with Nyström-based scalability. The authors provide convergence guarantees showing when collaboration improves estimation, and they present a GPU- and distribution-friendly implementation with POOL as a graph-free baseline. Empirical results on synthetic graphs demonstrate that GRULSIF (and its POOL variant) outperform state-of-the-art nodewise LRE methods, especially under limited local data and stronger task similarity, validating the theoretical predictions and practical utility for tasks like transfer learning, change-point detection, and hypothesis testing in networked settings.

Abstract

Assuming we have iid observations from two unknown probability density functions (pdfs), $p$ and $q$, the likelihood-ratio estimation (LRE) is an elegant approach to compare the two pdfs only by relying on the available data. In this paper, we introduce the first -to the best of our knowledge-graph-based extension of this problem, which reads as follows: Suppose each node $v$ of a fixed graph has access to observations coming from two unknown node-specific pdfs, $p_v$ and $q_v$, and the goal is to estimate for each node the likelihood-ratio between both pdfs by also taking into account the information provided by the graph structure. The node-level estimation tasks are supposed to exhibit similarities conveyed by the graph, which suggests that the nodes could collaborate to solve them more efficiently. We develop this idea in a concrete non-parametric method that we call Graph-based Relative Unconstrained Least-squares Importance Fitting (GRULSIF). We derive convergence rates for our collaborative approach that highlights the role played by variables such as the number of available observations per node, the size of the graph, and how accurately the graph structure encodes the similarity between tasks. These theoretical results explicit the situations where collaborative estimation effectively leads to an improvement in performance compared to solving each problem independently. Finally, in a series of experiments, we illustrate how GRULSIF infers the likelihood-ratios at the nodes of the graph more accurately compared to state-of-the art LRE methods, which would operate independently at each node, and we also verify that the behavior of GRULSIF is aligned with our previous theoretical analysis.

Collaborative likelihood-ratio estimation over graphs

TL;DR

The paper extends likelihood-ratio estimation to graphs by introducing GRULSIF, a non-parametric, graph-regularized method that learns node-specific relative likelihood-ratios in a shared VV-RKHS. It combines a nodewise χ^2-based loss with a Laplacian penalty to exploit graph smoothness, and leverages a Representer theorem to yield a tractable finite-dimensional optimization solved via Cyclic Block Coordinate Gradient Descent, with Nyström-based scalability. The authors provide convergence guarantees showing when collaboration improves estimation, and they present a GPU- and distribution-friendly implementation with POOL as a graph-free baseline. Empirical results on synthetic graphs demonstrate that GRULSIF (and its POOL variant) outperform state-of-the-art nodewise LRE methods, especially under limited local data and stronger task similarity, validating the theoretical predictions and practical utility for tasks like transfer learning, change-point detection, and hypothesis testing in networked settings.

Abstract

Assuming we have iid observations from two unknown probability density functions (pdfs), and , the likelihood-ratio estimation (LRE) is an elegant approach to compare the two pdfs only by relying on the available data. In this paper, we introduce the first -to the best of our knowledge-graph-based extension of this problem, which reads as follows: Suppose each node of a fixed graph has access to observations coming from two unknown node-specific pdfs, and , and the goal is to estimate for each node the likelihood-ratio between both pdfs by also taking into account the information provided by the graph structure. The node-level estimation tasks are supposed to exhibit similarities conveyed by the graph, which suggests that the nodes could collaborate to solve them more efficiently. We develop this idea in a concrete non-parametric method that we call Graph-based Relative Unconstrained Least-squares Importance Fitting (GRULSIF). We derive convergence rates for our collaborative approach that highlights the role played by variables such as the number of available observations per node, the size of the graph, and how accurately the graph structure encodes the similarity between tasks. These theoretical results explicit the situations where collaborative estimation effectively leads to an improvement in performance compared to solving each problem independently. Finally, in a series of experiments, we illustrate how GRULSIF infers the likelihood-ratios at the nodes of the graph more accurately compared to state-of-the art LRE methods, which would operate independently at each node, and we also verify that the behavior of GRULSIF is aligned with our previous theoretical analysis.
Paper Structure (29 sections, 13 theorems, 109 equations, 9 figures, 2 tables, 3 algorithms)

This paper contains 29 sections, 13 theorems, 109 equations, 9 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries and problem statement
  3. Problem statement
  4. Important notions for LRE
  5. Important notions for non-parametric estimation
  6. Important notions of Graph Signal Processing
  7. Graph-based Relative Unconstrained Least-squares Importance Fitting (GRULSIF)
  8. LRE and $\phi$-divergence estimation via Laplacian-penalized least-squares
  9. Comments regarding other $\phi$-divergences
  10. Convergence guarantees
  11. Practical implementation
  12. Computing the node parameter updates via CBCGD
  13. Nyström dimensionality reduction strategy
  14. POOL: a no graph variant
  15. Hyperparameters selection
  16. ...and 14 more sections

Key Result

Lemma 1

(Lemma 1 in Nguyen2007). For any class of functions $\mathcal{F} : \mathcal{X} \rightarrow \mathbb{R}$, the lower-bound for the similarity between two probability measures, $P$ and $Q$, admiting pdfs $p$ and $q$ with respect to the Lebuesgue measure is: where $\phi^{\star}$ denotes the convex conjugate of $\phi: \mathbb{R} \rightarrow \mathbb{R}$. The equality Eq. eq:variational_formulation_1 hol

Figures (9)

  • Figure 1: Likelihood-ratio estimation over a graph $G$. Simple example of the problem addressed by GRULSIF. Given different data points from two probabilistic models $p_v$ (blue) and $q_v$ (pink) at each node of $G$ (left-side figure), we aim to estimate the associated relative-likelihood ratio $r_v^{\alpha}$ (right-side figure) in a collaborative and distributed manner. In this example, the input domain of the data is $\mathcal{X}=\mathbb{R}^2$, and it is easy to see how any given $x\in{\mathcal{X}}$ gets essentially mapped to the graph signal $\mathbf{r}^{\alpha}(x) = (r^{\alpha}_1(x),...,r^{\alpha}_N(x))^{{\mkern-1.5mu\mathsf{T}}}$.
  • Figure 2: Experiment Synth.Ia
  • Figure 3: Experiment Synth.Ib
  • Figure 4: Experiment Synth.IIa
  • Figure 5: Experiment Synth.IIb
  • ...and 4 more figures

Theorems & Definitions (16)

  • Lemma 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Definition 5
  • Definition 6
  • Lemma 7
  • Lemma 8
  • Definition 9
  • Theorem 10
  • ...and 6 more