Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Fairness to Infinity: Outcome-Indistinguishable (Omni)Prediction in Evolving Graphs

Cynthia Dwork, Chris Hays, Nicole Immorlica, Juan C. Perdomo, Pranay Tankala

TL;DR

This work observes that, by combining a slightly modified form of the online K29 star algorithm of Vovk (2007) with basic facts from the theory of reproducing kernel Hilbert spaces, one can derive simple and efficient online algorithms satisfying outcome indistinguishability and omniprediction, with guarantees that improve upon, or are complementary to, those currently known.

Abstract

Professional networks provide invaluable entree to opportunity through referrals and introductions. A rich literature shows they also serve to entrench and even exacerbate a status quo of privilege and disadvantage. Hiring platforms, equipped with the ability to nudge link formation, provide a tantalizing opening for beneficial structural change. We anticipate that key to this prospect will be the ability to estimate the likelihood of edge formation in an evolving graph. Outcome-indistinguishable prediction algorithms ensure that the modeled world is indistinguishable from the real world by a family of statistical tests. Omnipredictors ensure that predictions can be post-processed to yield loss minimization competitive with respect to a benchmark class of predictors for many losses simultaneously, with appropriate post-processing. We begin by observing that, by combining a slightly modified form of the online K29 star algorithm of Vovk (2007) with basic facts from the theory of reproducing kernel Hilbert spaces, one can derive simple and efficient online algorithms satisfying outcome indistinguishability and omniprediction, with guarantees that improve upon, or are complementary to, those currently known. This is of independent interest. We apply these techniques to evolving graphs, obtaining online outcome-indistinguishable omnipredictors for rich -- possibly infinite -- sets of distinguishers that capture properties of pairs of nodes, and their neighborhoods. This yields, inter alia, multicalibrated predictions of edge formation with respect to pairs of demographic groups, and the ability to simultaneously optimize loss as measured by a variety of social welfare functions.

From Fairness to Infinity: Outcome-Indistinguishable (Omni)Prediction in Evolving Graphs

TL;DR

This work observes that, by combining a slightly modified form of the online K29 star algorithm of Vovk (2007) with basic facts from the theory of reproducing kernel Hilbert spaces, one can derive simple and efficient online algorithms satisfying outcome indistinguishability and omniprediction, with guarantees that improve upon, or are complementary to, those currently known.

Abstract

Professional networks provide invaluable entree to opportunity through referrals and introductions. A rich literature shows they also serve to entrench and even exacerbate a status quo of privilege and disadvantage. Hiring platforms, equipped with the ability to nudge link formation, provide a tantalizing opening for beneficial structural change. We anticipate that key to this prospect will be the ability to estimate the likelihood of edge formation in an evolving graph. Outcome-indistinguishable prediction algorithms ensure that the modeled world is indistinguishable from the real world by a family of statistical tests. Omnipredictors ensure that predictions can be post-processed to yield loss minimization competitive with respect to a benchmark class of predictors for many losses simultaneously, with appropriate post-processing. We begin by observing that, by combining a slightly modified form of the online K29 star algorithm of Vovk (2007) with basic facts from the theory of reproducing kernel Hilbert spaces, one can derive simple and efficient online algorithms satisfying outcome indistinguishability and omniprediction, with guarantees that improve upon, or are complementary to, those currently known. This is of independent interest. We apply these techniques to evolving graphs, obtaining online outcome-indistinguishable omnipredictors for rich -- possibly infinite -- sets of distinguishers that capture properties of pairs of nodes, and their neighborhoods. This yields, inter alia, multicalibrated predictions of edge formation with respect to pairs of demographic groups, and the ability to simultaneously optimize loss as measured by a variety of social welfare functions.

Paper Structure

This paper contains 69 sections, 40 theorems, 235 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Our contributions and related work.
  3. Relation to the graph prediction literature.
  4. Comparison with previous work in algorithmic fairness.
  5. Paper organization.
  6. Overview of technical results.
  7. Online outcome indistinguishability dwork2021outcome.
  8. RKHSs, the $\text{K29}^*$ algorithm, and the Any Kernel algorithm.
  9. Online omniprediction results.
  10. Other results.
  11. The Link Prediction Problem
  12. Data.
  13. Prediction protocol.
  14. Notation.
  15. Formal desiderata.
  16. ...and 54 more sections

Key Result

Theorem 1.1

Let $k$ be any kernel function and let ${\cal F}$ be its associated RKHS. Then, the Any Kernel algorithm generates a sequence of predictions $p_t\sim \Delta_t$ such that for any $f \in {\cal F}$: The second inequality holds if $k((x_t, p_t),(x_t,p_t)) \leqslant B^2$ for all $t$. Here, $\| f\|_{\mathcal{F}}$ is the norm of $f$ in ${\cal F}$ and the expectations are taken over the distributions $\D

Figures (3)

  • Figure 1: Pseudocode for the Any Kernel algorithm. Steps 1-3 are as in vovk2007non. Step 4 is inspired by foster2021forecast. In each iteration, solve the binary search problems in steps 3 or 4 using at most $\log(1/\varepsilon_t)$ oracle evaluations of $S_t$. Each evaluation of $S_t$ requires $t$ evaluations of the kernel $k$, hence the runtime at round $t$ is $\widetilde{{\cal O}}(t \cdot \mathsf{time}_k )$. If $k$ is forecast-continuous $\Delta_t$ is just a point mass at $p_t$. Otherwise, $\Delta_t$ is near deterministic: it is supported on just 2 points $q_t,q_t'$ which are very close together, $|q - q'|\leqslant {\cal O}(t^{-3})$. See \ref{['thm:indistinguishability_main']} for formal guarantees.
  • Figure 2: Extension of $\text{Any Kernel algorithm}$ for quantiles. The algorithm is essentially identical to the $\text{Any Kernel algorithm}$, except that the $S_t$ function has been defined slightly differently. As before, the algorithm is near-deterministic. The distribution $\Delta_t$ is either a point mass, or supported on two points that are very close together.
  • Figure 3: Extension of the Any Kernel algorithm for high-dimensional prediction. For simplicity, we state the algorithm assuming that the variational inequalities are solved exactly. However, as illustrated previously in quantile and binary prediction, the analysis can be easily modified to accomodate approximate solutions. The behavior of the algorithm for continuous kernels is the same as in vovk2005defensive. The extension to the discontinuous case is new.

Theorems & Definitions (93)

  • Theorem 1.1: Informal
  • Proposition 1.2: Informal
  • Lemma 1.3: Informal
  • Theorem 1.4: Informal
  • Definition 2.1
  • Definition 2.2
  • Definition 3.1
  • Theorem 3.2
  • proof
  • Corollary 3.3: Low-degree functions on $\{ 0, 1\}^n$
  • ...and 83 more