Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Nonconvexity of Push-Forward Constraints and Its Consequences in Machine Learning

Lucas de Lara, Mathis Deronzier, Alberto González-Sanz, Virgile Foy

TL;DR

This paper tackles the problem that push-forward constraints, essential in optimal transport, generative modeling, and group fairness, are typically nonconvex, precluding universal convex formulations. It develops a detailed convexity analysis of two key sets: transport maps $\mathcal{T}(P,Q)$ and equalizing maps $\mathcal{E}(P,Q)$, showing nonconvexity in broad regimes and highlighting the mismatch between convexity in measure space and function space. The authors then illustrate the practical impact on ML tasks, arguing that convex losses cannot quantify deviation from nonconvex push-forward constraints, with concrete implications for generative modeling and fairness. Finally, they propose two directions to recover convexity: weakening/strengthening the constraint to convex surrogates, or replacing deterministic maps with random couplings (Kantorovich-style relaxations), offering a roadmap for designing tractable, if not strictly convex, learning problems under push-forward conditions.

Abstract

The push-forward operation enables one to redistribute a probability measure through a deterministic map. It plays a key role in statistics and optimization: many learning problems (notably from optimal transport, generative modeling, and algorithmic fairness) include constraints or penalties framed as push-forward conditions on the model. However, the literature lacks general theoretical insights on the (non)convexity of such constraints and its consequences on the associated learning problems. This paper aims at filling this gap. In the first part, we provide a range of sufficient and necessary conditions for the (non)convexity of two sets of functions: the maps transporting one probability measure to another and the maps inducing equal output distributions across distinct probability measures. This highlights that for most probability measures, these push-forward constraints are not convex. In the second part, we show how this result implies critical limitations on the design of convex optimization problems for learning generative models or groupwise fair predictors. This work will hopefully help researchers and practitioners have a better understanding of the critical impact of push-forward conditions onto convexity.

On the Nonconvexity of Push-Forward Constraints and Its Consequences in Machine Learning

TL;DR

This paper tackles the problem that push-forward constraints, essential in optimal transport, generative modeling, and group fairness, are typically nonconvex, precluding universal convex formulations. It develops a detailed convexity analysis of two key sets: transport maps and equalizing maps , showing nonconvexity in broad regimes and highlighting the mismatch between convexity in measure space and function space. The authors then illustrate the practical impact on ML tasks, arguing that convex losses cannot quantify deviation from nonconvex push-forward constraints, with concrete implications for generative modeling and fairness. Finally, they propose two directions to recover convexity: weakening/strengthening the constraint to convex surrogates, or replacing deterministic maps with random couplings (Kantorovich-style relaxations), offering a roadmap for designing tractable, if not strictly convex, learning problems under push-forward conditions.

Abstract

The push-forward operation enables one to redistribute a probability measure through a deterministic map. It plays a key role in statistics and optimization: many learning problems (notably from optimal transport, generative modeling, and algorithmic fairness) include constraints or penalties framed as push-forward conditions on the model. However, the literature lacks general theoretical insights on the (non)convexity of such constraints and its consequences on the associated learning problems. This paper aims at filling this gap. In the first part, we provide a range of sufficient and necessary conditions for the (non)convexity of two sets of functions: the maps transporting one probability measure to another and the maps inducing equal output distributions across distinct probability measures. This highlights that for most probability measures, these push-forward constraints are not convex. In the second part, we show how this result implies critical limitations on the design of convex optimization problems for learning generative models or groupwise fair predictors. This work will hopefully help researchers and practitioners have a better understanding of the critical impact of push-forward conditions onto convexity.
Paper Structure (21 sections, 13 theorems, 37 equations)

This paper contains 21 sections, 13 theorems, 37 equations.

Table of Contents

  1. Introduction
  2. Outline
  3. Preliminaries
  4. Notations and definitions
  5. Push-forward calculus
  6. The shape of push-forward constraints
  7. Transport maps
  8. Equalizing maps
  9. Application to machine learning
  10. Learning and convexity
  11. Learning under push-forward constraints or penalties
  12. Generative modeling
  13. Group-level algorithmic fairness
  14. Toward recovering convexity
  15. Weakening or strengthening the condition
  16. ...and 6 more sections

Key Result

Proposition 2.1

Let $f,g \in \mathcal{G}$, $\mu,\nu \in \mathcal{M}(\mathbb{R}^d)$, and $s \in \mathbb{R}$. The following properties hold:

Theorems & Definitions (21)

  • Proposition 2.1: basic push-forward calculus
  • Example 3.1: simple transport maps
  • Theorem 3.2: nowhere convexity of the set of squared-norm matching functions
  • Corollary 3.3: nonconvexity of the set of transport maps
  • Proposition 3.4: transport maps for a continuous source measure
  • Remark 3.5: existence and uniqueness of monotone push-forward maps
  • Proposition 3.6: transport maps between uniform finitely supported measures
  • Proposition 3.7: basic properties
  • Example 3.8: simple equalizing maps
  • Proposition 3.9: equalizing maps between continuous measures
  • ...and 11 more