Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integrating Unstructured Text into Causal Inference: Empirical Evidence from Real Data

Boning Zhou, Ziyu Wang, Han Hong, Haoqi Hu

TL;DR

This paper addresses the challenge of performing causal inference when structured tabular data are unavailable by leveraging unstructured text with transformer-based language models. It builds an Interactive Regression Model (IRM) and employs cross-fitting, doubly robust labeling, and best linear predictor concepts to estimate ATE, ATET, GATE, and CATE from text and from numeric data, enabling direct comparison on real-world data. Using an anti-counterfeit campaign dataset, the authors show that text-derived causal estimates are highly consistent with those from structured data across population, group, and individual levels, with lift curves indicating practical predictive utility. The work demonstrates the viability of text-only causal inference for timely, data-driven business decisions and lays groundwork for future extensions to continuous or multi-treatment scenarios and multi-modal training.

Abstract

Causal inference, a critical tool for informing business decisions, traditionally relies heavily on structured data. However, in many real-world scenarios, such data can be incomplete or unavailable. This paper presents a framework that leverages transformer-based language models to perform causal inference using unstructured text. We demonstrate the effectiveness of our framework by comparing causal estimates derived from unstructured text against those obtained from structured data across population, group, and individual levels. Our findings show consistent results between the two approaches, validating the potential of unstructured text in causal inference tasks. Our approach extends the applicability of causal inference methods to scenarios where only textual data is available, enabling data-driven business decision-making when structured tabular data is scarce.

Integrating Unstructured Text into Causal Inference: Empirical Evidence from Real Data

TL;DR

This paper addresses the challenge of performing causal inference when structured tabular data are unavailable by leveraging unstructured text with transformer-based language models. It builds an Interactive Regression Model (IRM) and employs cross-fitting, doubly robust labeling, and best linear predictor concepts to estimate ATE, ATET, GATE, and CATE from text and from numeric data, enabling direct comparison on real-world data. Using an anti-counterfeit campaign dataset, the authors show that text-derived causal estimates are highly consistent with those from structured data across population, group, and individual levels, with lift curves indicating practical predictive utility. The work demonstrates the viability of text-only causal inference for timely, data-driven business decisions and lays groundwork for future extensions to continuous or multi-treatment scenarios and multi-modal training.

Abstract

Causal inference, a critical tool for informing business decisions, traditionally relies heavily on structured data. However, in many real-world scenarios, such data can be incomplete or unavailable. This paper presents a framework that leverages transformer-based language models to perform causal inference using unstructured text. We demonstrate the effectiveness of our framework by comparing causal estimates derived from unstructured text against those obtained from structured data across population, group, and individual levels. Our findings show consistent results between the two approaches, validating the potential of unstructured text in causal inference tasks. Our approach extends the applicability of causal inference methods to scenarios where only textual data is available, enabling data-driven business decision-making when structured tabular data is scarce.
Paper Structure (14 sections, 1 theorem, 35 equations, 9 figures, 3 tables, 1 algorithm)

This paper contains 14 sections, 1 theorem, 35 equations, 9 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Causal Estimation
  3. Model
  4. Estimation
  5. Algorithm
  6. Experiment Design
  7. Data
  8. Model Training
  9. Results
  10. Conclusion & Extension
  11. Derivation of CATE
  12. Metric definition for Table \ref{['table: numerical comparison']}
  13. Graph for Corr T and Corr C
  14. GATEs for Different Groups

Key Result

Theorem A.2

Given $\tilde{\eta}$ with assuming $\tilde{g}(1, Z)$ and $\tilde{g}(0, Z)$ have finite second moment and do not equal to each other almost surely. Then $(a_1, b_1)$ by considering following linear projection, with $\; E[\epsilon \tilde{X}] = 0\;, \tilde{X} = [1, \tilde{\theta}(Z) - E[\tilde{\theta}(Z)]]'$ has the following expression: With $bias_1$ and $bias_2$ are two random variable:

Figures (9)

  • Figure 1: Illustrative example of unstructured texts
  • Figure 2: ATE & ATT Comparison: Box plots
  • Figure 3: CATE Comparison: CATE Curves
  • Figure 4: CATE Comparison: Lift Curves
  • Figure 5: Gradient Boost Control
  • ...and 4 more figures

Theorems & Definitions (4)

  • Definition A.1
  • Theorem A.2
  • proof
  • Remark A.3