Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Unstructured Data to Demand Counterfactuals: Theory and Practice

Timothy Christensen, Giovanni Compiani

TL;DR

The paper tackles bias in counterfactual demand analysis when high-dimensional proxies (e.g., embeddings from unstructured data) imperfectly capture latent product differentiation. It develops a post-estimation bias-correction that uses a composite parameter $\gamma(\theta,e)$ to orthogonalize the effect of proxy mismeasurement on counterfactuals, yielding unbiased, efficient estimates with closed-form standard errors. The framework applies to both BLP-type market-level models with endogenous prices and microdata with product fixed effects, and it accommodates data-dependent proxies, including embeddings from fine-tuned ML models. Complementary LM-based diagnostics (LM$_1$ and LM$_2$) guide proxy selection and dimensionality, while simulations and an empirical application demonstrate substantial gains in predicting counterfactual substitution and identifying effective proxies. Overall, the toolkit offers a practical, scalable approach to robust counterfactual inference in demand estimation with unstructured data.

Abstract

Empirical models of demand for differentiated products rely on low-dimensional product representations to capture substitution patterns. These representations are increasingly proxied by applying ML methods to high-dimensional, unstructured data, including product descriptions and images. When proxies fail to capture the true dimensions of differentiation that drive substitution, standard workflows will deliver biased counterfactuals and invalid inference. We develop a practical toolkit that corrects this bias and ensures valid inference for a broad class of counterfactuals. Our approach applies to market-level and/or individual data, requires minimal additional computation, is efficient, delivers simple formulas for standard errors, and accommodates data-dependent proxies, including embeddings from fine-tuned ML models. It can also be used with standard quantitative attributes when mismeasurement is a concern. In addition, we propose diagnostics to assess the adequacy of the proxy construction and dimension. The approach yields meaningful improvements in predicting counterfactual substitution in both simulations and an empirical application.

From Unstructured Data to Demand Counterfactuals: Theory and Practice

TL;DR

The paper tackles bias in counterfactual demand analysis when high-dimensional proxies (e.g., embeddings from unstructured data) imperfectly capture latent product differentiation. It develops a post-estimation bias-correction that uses a composite parameter to orthogonalize the effect of proxy mismeasurement on counterfactuals, yielding unbiased, efficient estimates with closed-form standard errors. The framework applies to both BLP-type market-level models with endogenous prices and microdata with product fixed effects, and it accommodates data-dependent proxies, including embeddings from fine-tuned ML models. Complementary LM-based diagnostics (LM and LM) guide proxy selection and dimensionality, while simulations and an empirical application demonstrate substantial gains in predicting counterfactual substitution and identifying effective proxies. Overall, the toolkit offers a practical, scalable approach to robust counterfactual inference in demand estimation with unstructured data.

Abstract

Empirical models of demand for differentiated products rely on low-dimensional product representations to capture substitution patterns. These representations are increasingly proxied by applying ML methods to high-dimensional, unstructured data, including product descriptions and images. When proxies fail to capture the true dimensions of differentiation that drive substitution, standard workflows will deliver biased counterfactuals and invalid inference. We develop a practical toolkit that corrects this bias and ensures valid inference for a broad class of counterfactuals. Our approach applies to market-level and/or individual data, requires minimal additional computation, is efficient, delivers simple formulas for standard errors, and accommodates data-dependent proxies, including embeddings from fine-tuned ML models. It can also be used with standard quantitative attributes when mismeasurement is a concern. In addition, we propose diagnostics to assess the adequacy of the proxy construction and dimension. The approach yields meaningful improvements in predicting counterfactual substitution in both simulations and an empirical application.
Paper Structure (31 sections, 9 theorems, 117 equations, 6 figures)

This paper contains 31 sections, 9 theorems, 117 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Case 1: Endogenous Prices
  3. Model and Data
  4. Bias-Corrected Counterfactuals
  5. Diagnostics
  6. Diagnostic 1: Is $\hat{\gamma}$ Close To $\gamma_0$?
  7. Diagnostic 2: Is The Dimension of $\tilde{e}$ Correct?
  8. Practitioner's Guide
  9. What About Models with Standard Numeric Attributes?
  10. Case 2: Individual-Level Price Variation and Product Fixed Effects
  11. Model and Data
  12. Bias-Corrected Counterfactuals
  13. Diagnostics
  14. Diagnostic 1: Is $\hat{\gamma}$ Close To $\gamma_0$?
  15. Diagnostic 2: Is The Dimension of $\tilde{e}$ Correct?
  16. ...and 16 more sections

Key Result

Proposition 1

Let Assumption a.2 hold and $\hat{\gamma} = \gamma_0 + o_p(T^{-1/4})$. Then $\sqrt T(\hat{\kappa}_{bc} - \kappa_0)$ converges in distribution ($\mathcal{M}$-stably) as $T \to \infty$ to a mixed Gaussian random variable with mean zero and $\mathcal{M}$-measurable variance

Figures (6)

  • Figure 1: Distribution of the naive and bias-corrected estimators
  • Figure 2: Bias and RMSE of the naive and bias-corrected estimators
  • Figure 3: Distance between $\hat{\gamma}$ and $\gamma_0$ versus $LM_1$ diagnostic
  • Figure 4: Rates of correct closest substitutes predictions.
  • Figure 5: $LM_1$ diagnostic in empirical application
  • ...and 1 more figures

Theorems & Definitions (28)

  • Example 1: BLP
  • Example 2: Micro BLP
  • Remark 1
  • Remark 2: Easy to compute standard errors
  • Remark 3: Efficiency
  • Remark 4: Fine Tuning
  • Example 3: Mixed Logit with Fixed Effects
  • Remark 5: Easy to compute standard errors
  • Remark 6: Semiparametric Efficiency
  • Remark 7: Fine Tuning
  • ...and 18 more