Constrained Reweighting of Distributions: an Optimal Transport Approach

TL;DR

The paper tackles reweighting empirical distributions under flexible distributional constraints by marrying maximum entropy with optimal transport, specifically using the $W_2$ distance to a target distribution. It develops a distribution-guided constrained entropy framework that yields a discrete, weight-adjusted empirical measure closely aligned with a (potentially continuous) target, with departures controlled by a tuning parameter. The authors instantiate the approach in three domains: semi-parametric inference for complex surveys (via bootstrapped distributionally constrained models and ETEL), demographic parity in machine learning (through in-model and two-step fairness schemes leveraging $W_2$ constraints), and entropy-based portfolio optimization (using Skew-normal targets to balance diversification and fidelity). Across these applications, the method demonstrates improved inference accuracy, fairness, and diversification trade-offs, while providing a versatile, interpretable framework for data re-weighting under side constraints.

Abstract

We commonly encounter the problem of identifying an optimally weight adjusted version of the empirical distribution of observed data, adhering to predefined constraints on the weights. Such constraints often manifest as restrictions on the moments, tail behaviour, shapes, number of modes, etc., of the resulting weight adjusted empirical distribution. In this article, we substantially enhance the flexibility of such methodology by introducing a nonparametrically imbued distributional constraints on the weights, and developing a general framework leveraging the maximum entropy principle and tools from optimal transport. The key idea is to ensure that the maximum entropy weight adjusted empirical distribution of the observed data is close to a pre-specified probability distribution in terms of the optimal transport metric while allowing for subtle departures. The versatility of the framework is demonstrated in the context of three disparate applications where data re-weighting is warranted to satisfy side constraints on the optimization problem at the heart of the statistical task: namely, portfolio allocation, semi-parametric inference for complex surveys, and ensuring algorithmic fairness in machine learning algorithms.

Figures (6)

• Figure 1: Distress Analysis Interview Corpus. Empirical cdfs of fitted $h$ for the two groups, with no fairness constraint $( W_{2} = 19.32)$, fair post-processing $( W_{2} = 2.24)$, and fair model fitting with $( W_{2}= 0.79$) respectively at $\lambda^{\star} = 0$.
• Figure 2: Distress Analysis Interview Corpus. Maximum likelihood estimates of the regression coefficients under both two-step and in model schemes. In the in model scheme the estimates get slightly modified since the regression coefficients and the weights assigned to the data are learned simultaneously. For details on the in model and two-step approaches, refer to equations \ref{['wgf_simultenous']} and \ref{['wgf_twostep_1']}-\ref{['wgf_twostep_2']} respectively.
• Figure 3: COMPAS dataset. Empirical cdfs of fitted $h$ for the two groups, with no fairness constraint $( W_{2} = 0.72)$, fair post-processing $( W_{2} = 0.05)$, and fair model fitting with $( W_{2}= 0.02)$ respectively at $\lambda^{\star} = 0$.
• Figure 4: COMPAS dataset. Maximum likelihood estimates of the regression coefficients under both two-step and in model schemes. In the in model scheme the estimates get slightly modified since the regression coefficients and the weights assigned to the data are learned simultaneously.
• Figure 5: Limitations of mean-variance optimal portfolio: (i) The skewness and excess kurtosis plots provide evidence that the normality assumption for expected returns does not hold. (ii) Small value of $\lambda$ leads to zero weight to several assets.

Theorems & Definitions (4)

• Definition 1
• Definition 2: Demographic parity, 90450a4b5b49471b8111fc88355f2e7f
• Definition 3: Demographic parity in expectation, 90450a4b5b49471b8111fc88355f2e7f
• Definition 4: Demographic parity in Wasserstein metric