Reading Between the Lines: Deconfounding Causal Estimates using Text Embeddings and Deep Learning

TL;DR

This paper tackles unobserved confounding in observational causal inference by leveraging high-dimensional unstructured text as proxies for latent variables. It introduces Neural Network-Enhanced Double Machine Learning to model nuisance functions on embeddings, addressing an Architecture Gap between tree-based methods and the smooth geometry of embedding spaces. Using a rigorous synthetic benchmark with known ground-truth effects, it shows that standard tree-based DML leaves substantial bias (around +24%), while neural DML reduces bias to about -0.86% with optimized, parsimonious architectures. Sectoral analyses and robustness checks demonstrate the neural approach consistently identifies the true causal effect, underscoring the practical importance of neural nuisance learners for causal inference with text data.

Abstract

Estimating causal treatment effects in observational settings is frequently compromised by selection bias arising from unobserved confounders. While traditional econometric methods struggle when these confounders are orthogonal to structured covariates, high-dimensional unstructured text often contains rich proxies for these latent variables. This study proposes a Neural Network-Enhanced Double Machine Learning (DML) framework designed to leverage text embeddings for causal identification. Using a rigorous synthetic benchmark, we demonstrate that unstructured text embeddings capture critical confounding information that is absent from structured tabular data. However, we show that standard tree-based DML estimators retain substantial bias (+24%) due to their inability to model the continuous topology of embedding manifolds. In contrast, our deep learning approach reduces bias to -0.86% with optimized architectures, effectively recovering the ground-truth causal parameter. These findings suggest that deep learning architectures are essential for satisfying the unconfoundedness assumption when conditioning on high-dimensional natural language data

Figures (6)

• Figure 1: Structural Causal Model as a DAG. The diagram maps the coefficients from the structural equations to the causal graph. The solid lines represent relationships captured by observed covariates $X$ ($\delta, \beta$). The red dashed lines represent the unobserved influence of Ability ($U$), governed by the selection parameter $\eta$ and the outcome parameter $\gamma$. The open backdoor path flowing through $U$ generates the bias term derived in Equation (3).
• Figure 2: Directed Acyclic Graph (DAG) of the Proxy Identification Strategy. The dashed node $U$ represents latent confounders which are unobserved in the structured data $X$. However, $U$ causally influences the unstructured text $W$ (solid line). By conditioning on $W$, the estimator blocks the confounding influence of $U$ on the Treatment ($T$) and Outcome ($Y$).
• Figure 3: Decomposition of Identification Strategy.Panel A: Selection bias in ability. Panel B: Scatter plot demonstrating confounding and common support. Panel C: The first principal component of text embeddings plotted against latent ability ($r=-0.85$). Panel D: Comparison of explained variance ($R^2$) in latent ability across covariate sets.
• Figure 4: Estimator Performance by Sector. The grouped bar chart compares the estimated monthly earnings effect across five professional sectors. In every domain, the Neural Network (Gold) aligns closest to the True Effect (Dark Grey). Notably, in Data Science, the Tree-based model (Green) under-corrects, while in Web Development it over-corrects; the Neural Network consistently minimizes these residual biases.
• Figure 5: The Bias-Variance Trade-off in Nuisance Learners ($n=10$ seeds). The boxplots visualize the distribution of ATE estimates across 10 independent runs. The tree-based models (Gray/Blue) are "precisely wrong"—stable but biased far from the Truth (dashed green line). The Neural Network (Yellow) is "approximately right," exhibiting higher variance but successfully covering the true parameter value.