Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Riesz Representer Fitting under Bregman Divergence: A Unified Framework for Debiased Machine Learning

Masahiro Kato

TL;DR

This work develops a unified framework for estimating the Riesz representer under Bregman divergence, linking Riesz regression, covariate balancing, and density ratio methods. It shows that squared loss corresponds to Riesz regression while KL-type losses correspond to tailored loss with entropy balancing duals, and it introduces automatic covariate balancing through generalized linear representations. The authors establish convergence rates for RKHS and neural-network models and derive an efficient AIPW-type estimator for θ0, with applications to ATE, AME, APE, and covariate shift. Empirical results on synthetic and semi-synthetic data demonstrate competitive performance and stable inference across divergence choices, highlighting the framework’s practical relevance for debiased machine learning in causal and structural estimation.

Abstract

Estimating the Riesz representer is a central problem in debiased machine learning for causal and structural parameter estimation. Various methods for Riesz representer estimation have been proposed, including Riesz regression and covariate balancing. This study unifies these methods within a single framework. Our framework fits a Riesz representer model to the true Riesz representer under a Bregman divergence, which includes the squared loss and the Kullback--Leibler (KL) divergence as special cases. We show that the squared loss corresponds to Riesz regression, and the KL divergence corresponds to tailored loss minimization, where the dual solutions correspond to stable balancing weights and entropy balancing weights, respectively, under specific model specifications. We refer to our method as generalized Riesz regression, and we refer to the associated duality as automatic covariate balancing. Our framework also generalizes density ratio fitting under a Bregman divergence to Riesz representer estimation, and it includes various applications beyond density ratio estimation. We also provide a convergence analysis for both cases where the model class is a reproducing kernel Hilbert space (RKHS) and where it is a neural network.

Riesz Representer Fitting under Bregman Divergence: A Unified Framework for Debiased Machine Learning

TL;DR

This work develops a unified framework for estimating the Riesz representer under Bregman divergence, linking Riesz regression, covariate balancing, and density ratio methods. It shows that squared loss corresponds to Riesz regression while KL-type losses correspond to tailored loss with entropy balancing duals, and it introduces automatic covariate balancing through generalized linear representations. The authors establish convergence rates for RKHS and neural-network models and derive an efficient AIPW-type estimator for θ0, with applications to ATE, AME, APE, and covariate shift. Empirical results on synthetic and semi-synthetic data demonstrate competitive performance and stable inference across divergence choices, highlighting the framework’s practical relevance for debiased machine learning in causal and structural estimation.

Abstract

Estimating the Riesz representer is a central problem in debiased machine learning for causal and structural parameter estimation. Various methods for Riesz representer estimation have been proposed, including Riesz regression and covariate balancing. This study unifies these methods within a single framework. Our framework fits a Riesz representer model to the true Riesz representer under a Bregman divergence, which includes the squared loss and the Kullback--Leibler (KL) divergence as special cases. We show that the squared loss corresponds to Riesz regression, and the KL divergence corresponds to tailored loss minimization, where the dual solutions correspond to stable balancing weights and entropy balancing weights, respectively, under specific model specifications. We refer to our method as generalized Riesz regression, and we refer to the associated duality as automatic covariate balancing. Our framework also generalizes density ratio fitting under a Bregman divergence to Riesz representer estimation, and it includes various applications beyond density ratio estimation. We also provide a convergence analysis for both cases where the model class is a reproducing kernel Hilbert space (RKHS) and where it is a neural network.
Paper Structure (98 sections, 19 theorems, 296 equations, 1 figure, 3 tables)

This paper contains 98 sections, 19 theorems, 296 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Setup
  3. Riesz representer
  4. Neyman Orthogonal Scores
  5. Examples
  6. ATE estimation.
  7. AME estimation.
  8. APE estimation.
  9. Covariate shift adaptation.
  10. Notations and assumptions.
  11. Generalized Riesz Regression
  12. Bregman Divergence
  13. Special Cases of the Bregman Divergence
  14. SQ-Riesz Regression
  15. UKL-Riesz Regression
  16. ...and 83 more sections

Key Result

Theorem 4.1

Assume that there exists a function $\widetilde{g} \colon {\mathcal{X}} \times {\mathbb{R}} \to {\mathbb{R}}$ such that that is, $\partial g(\alpha_{{\bm{\beta}}}(X_i))$ is linear in ${\bm{\phi}}(X_i)^\top {\bm{\beta}}$. In this case, a Riesz representer model $\widehat{\alpha} = \alpha_{\widehat{{\bm{\beta}}}}$ trained by empirical risk minimization satisfies Generalized Riesz regression return

Figures (1)

  • Figure 1: A unified framework for debiased machine learning via Riesz representer estimation and Bregman divergence minimization.

Theorems & Definitions (26)

  • Theorem 4.1: Automatic Covariate Balancing
  • Theorem 5.1: $L_2$-norm estimation error bound
  • Definition 5.1: FNNs. From Zheng2022anerror
  • Theorem 5.2: Estimation error bound for neural networks
  • Theorem 5.3: Asymptotic normality
  • Remark : From density ratio estimation to covariate shift adaptation
  • Proposition B.1: From Theorem 2.1 in Bartlett2005localrademacher
  • Definition B.1
  • Proposition B.2: Talagrand's Lemma
  • Lemma C.1: $L_2$ distance bound from Lemma 4 in Kato2021nonnegativebregman
  • ...and 16 more