Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Heterogeneity Analysis with Heterogeneous Treatments

Phillip Heiler, Michael C. Knaus

TL;DR

A novel decomposition framework that disentangles contributions of effect heterogeneity and distinct components of treatment heterogeneity to observed group-level differences is developed and debiased machine learning estimators that adapt to many discrete and/or continuous treatments and limited overlap are proposed.

Abstract

Analysis of effect heterogeneity at the group level is standard practice in empirical treatment evaluation. However, treatments analyzed are often aggregates of multiple underlying treatments which are themselves heterogeneous, e.g. different modules of training programs. In these settings, conventional approaches such as comparing (adjusted) differences-in-means across groups can produce misleading conclusions when underlying treatment propensities differ systematically between groups. This paper develops a novel decomposition framework that disentangles contributions of effect heterogeneity and distinct components of treatment heterogeneity to observed group-level differences. We propose debiased machine learning estimators that adapt to many discrete and/or continuous treatments and limited overlap. We revisit a widely documented gender gap in training returns of an active labor market policy. The decomposition reveals that it is almost entirely driven by women being treated differently rather than different returns from identical treatments. In particular, women are disproportionately targeted towards vocational training tracks with lower unconditional returns.

Heterogeneity Analysis with Heterogeneous Treatments

TL;DR

A novel decomposition framework that disentangles contributions of effect heterogeneity and distinct components of treatment heterogeneity to observed group-level differences is developed and debiased machine learning estimators that adapt to many discrete and/or continuous treatments and limited overlap are proposed.

Abstract

Analysis of effect heterogeneity at the group level is standard practice in empirical treatment evaluation. However, treatments analyzed are often aggregates of multiple underlying treatments which are themselves heterogeneous, e.g. different modules of training programs. In these settings, conventional approaches such as comparing (adjusted) differences-in-means across groups can produce misleading conclusions when underlying treatment propensities differ systematically between groups. This paper develops a novel decomposition framework that disentangles contributions of effect heterogeneity and distinct components of treatment heterogeneity to observed group-level differences. We propose debiased machine learning estimators that adapt to many discrete and/or continuous treatments and limited overlap. We revisit a widely documented gender gap in training returns of an active labor market policy. The decomposition reveals that it is almost entirely driven by women being treated differently rather than different returns from identical treatments. In particular, women are disproportionately targeted towards vocational training tracks with lower unconditional returns.

Paper Structure

This paper contains 62 sections, 3 theorems, 129 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Literature
  3. Motivating Example
  4. Finding Heterogeneity Without Heterogeneous Effects
  5. Finding Heterogeneity When Group Effects Are Homogeneous
  6. Framework and Decomposition
  7. Double Aggregation Framework
  8. Remark 1.
  9. Application: Training Specializations in Job Corps
  10. Decomposing a Single Group Mean
  11. Decomposing a Single Group Difference-in-Means
  12. Decomposing Group Differences-in-Means
  13. Generic Implications for Heterogeneity Analyses
  14. Identification of Decomposition Parameters
  15. Extensions and Special Cases
  16. ...and 47 more sections

Key Result

Theorem 7.1

For any aggregation $a$, $g$, and bounded sequence vector $c_n \lesssim 1$ with $c_n \neq 0$, given Assumptions A.1 - A.6, we have that all decomposition parameters are asymptotically normal Moreover when $J \left(\frac{J}{n}\right)^{\left[\frac{1}{2}\wedge \frac{m}{2+m}\right]} = o(1)$, then asymptotic normality applies with $\Sigma$ replaced by its sample equivalent with estimated nuisances.

Figures (5)

  • Figure 1: Decomposition of Interaction Coefficient in Job Corps Application
  • Figure 2: Decomposition of Group Means in Job Corps Application
  • Figure 3: Decomposition of Gender Differences in Job Corps
  • Figure 4: Simulation of Finite Sample Power
  • Figure F.1: Decomposition of Adjusted Gender Differences in Job Corps

Theorems & Definitions (3)

  • Theorem 7.1: Large Sample Inference
  • Theorem 8.1
  • Corollary 8.1