Double Machine Learning meets Panel Data -- Promises, Pitfalls, and Potential Solutions

TL;DR

This paper explores how DML can be adapted for panel data in the presence of unobserved heterogeneity and finds that using predictive models based on the correlated random effects approach within DML leads to accurate coefficient estimates across settings, given a sample size that is large relative to the number of observed confounders.

Abstract

Estimating causal effect using machine learning (ML) algorithms can help to relax functional form assumptions if used within appropriate frameworks. However, most of these frameworks assume settings with cross-sectional data, whereas researchers often have access to panel data, which in traditional methods helps to deal with unobserved heterogeneity between units. In this paper, we explore how we can adapt double/debiased machine learning (DML) (Chernozhukov et al., 2018) for panel data in the presence of unobserved heterogeneity. This adaptation is challenging because DML's cross-fitting procedure assumes independent data and the unobserved heterogeneity is not necessarily additively separable in settings with nonlinear observed confounding. We assess the performance of several intuitively appealing estimators in a variety of simulations. While we find violations of the cross-fitting assumptions to be largely inconsequential for the accuracy of the effect estimates, many of the considered methods fail to adequately account for the presence of unobserved heterogeneity. However, we find that using predictive models based on the correlated random effects approach (Mundlak, 1978) within DML leads to accurate coefficient estimates across settings, given a sample size that is large relative to the number of observed confounders. We also show that the influence of the unobserved heterogeneity on the observed confounders plays a significant role for the performance of most alternative methods.

Figures (20)

• Figure 1: Causal graph for the assumed causal structure "unconfoundedness". $W_i$: treatment variable, $Y_i$: outcome variable, $\boldsymbol{X_i}$: observed confounding variables. The relationships between $\boldsymbol{X_i}$ and $W$ ($m_0()$), and $\boldsymbol{X_i}$ and $Y_i$ ($g_0()$), are potentially complex and nonlinear.
• Figure 2: Possible DGPs for panel data settings. $W_{it}$ (treatment), $Y_{it}$ (outcome), and $X_{it}$ (observed confounders) vary across both units and time. $U_{i}$ is unobserved unit-specific and time-constant heterogeneity. We consider three causal structures: (A)$U_i$ does not influence any other variables (or does not exist), (B)$U_i$ only influences $W_{it}$ and $Y_{it}$, (C)$U_i$ additionally influences $X_{it}$.
• Figure 3: Results for utilizing different cross-fitting techniques (Table \ref{['tab:split_DML_approaches']}) within various DML estimators. The vertical axis depicts the estimated coefficient. The dashed line marks the true causal effect ($\beta = 1$). The boxplots show the distribution of estimated coefficients across 100 simulated datasets for each method. Data is generated according to DGP (3), with one observed confounder, u-shaped functional forms and a large degree of autocorrelation ($\rho = 0.9$). NLO: neighbors-left-out cross-fitting.
• Figure 4: Results for our baseline simulation with $N=500$ units and $T = 10$ periods. The horizontal axis displays the different methods from Table \ref{['tab:methods_impl']}. The vertical axis depicts the estimated coefficient. The dashed line marks the true causal effect ($\beta = 1$). The boxplots show the distribution of estimated coefficients across 100 simulated datasets for each method. The three rows contain three different DGPs: "$no\, U$" indicates no unobserved heterogeneity, "$U\, |\, X$" means the unobserved heterogeneity influences treatment and outcome, but not confounders, and "$U \rightarrow X$" means the unobserved heterogeneity also influences the confounders.
• Figure 5: Results for the setting with $N=10$ units and $T = 500$ periods. The dashed line marks the true causal effect ($\beta = 1$). The boxplots show the distribution of estimated coefficients across 100 simulated datasets for each method. The three rows contain three different DGPs: "$no\, U$" indicates no unobserved heterogeneity, "$U\, |\, X$" means the unobserved heterogeneity influences treatment and outcome, but not confounders, and "$U \rightarrow X$" means the unobserved heterogeneity also influences the confounders.