Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Learning for Individual Heterogeneity

Max H. Farrell, Tengyuan Liang, Sanjog Misra

TL;DR

Deep neural networks are well-suited to structured modeling of heterogeneity: the network architecture can be designed to match the global structure of the economic model, giving novel methodology for deep learning as well as, more formally, improved rates of convergence.

Abstract

This paper integrates deep neural networks (DNNs) into structural economic models to increase flexibility and capture rich heterogeneity while preserving interpretability. Economic structure and machine learning are complements in empirical modeling, not substitutes: DNNs provide the capacity to learn complex, non-linear heterogeneity patterns, while the structural model ensures the estimates remain interpretable and suitable for decision making and policy analysis. We start with a standard parametric structural model and then enrich its parameters into fully flexible functions of observables, which are estimated using a particular DNN architecture whose structure reflects the economic model. We illustrate our framework by studying demand estimation in consumer choice. We show that by enriching a standard demand model we can capture rich heterogeneity, and further, exploit this heterogeneity to create a personalized pricing strategy. This type of optimization is not possible without economic structure, but cannot be heterogeneous without machine learning. Finally, we provide theoretical justification of each step in our proposed methodology. We first establish non-asymptotic bounds and convergence rates of our structural deep learning approach. Next, a novel and quite general influence function calculation allows for feasible inference via double machine learning in a wide variety of contexts. These results may be of interest in many other contexts, as they generalize prior work.

Deep Learning for Individual Heterogeneity

TL;DR

Deep neural networks are well-suited to structured modeling of heterogeneity: the network architecture can be designed to match the global structure of the economic model, giving novel methodology for deep learning as well as, more formally, improved rates of convergence.

Abstract

This paper integrates deep neural networks (DNNs) into structural economic models to increase flexibility and capture rich heterogeneity while preserving interpretability. Economic structure and machine learning are complements in empirical modeling, not substitutes: DNNs provide the capacity to learn complex, non-linear heterogeneity patterns, while the structural model ensures the estimates remain interpretable and suitable for decision making and policy analysis. We start with a standard parametric structural model and then enrich its parameters into fully flexible functions of observables, which are estimated using a particular DNN architecture whose structure reflects the economic model. We illustrate our framework by studying demand estimation in consumer choice. We show that by enriching a standard demand model we can capture rich heterogeneity, and further, exploit this heterogeneity to create a personalized pricing strategy. This type of optimization is not possible without economic structure, but cannot be heterogeneous without machine learning. Finally, we provide theoretical justification of each step in our proposed methodology. We first establish non-asymptotic bounds and convergence rates of our structural deep learning approach. Next, a novel and quite general influence function calculation allows for feasible inference via double machine learning in a wide variety of contexts. These results may be of interest in many other contexts, as they generalize prior work.

Paper Structure

This paper contains 34 sections, 3 theorems, 77 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Literature
  3. The Importance of Structure
  4. Embedding Deep Learning in Structural Models
  5. Enriched Structural Model
  6. Structural Deep Learning
  7. Inference
  8. Application: Advertising and Personalized Interest Rates
  9. Empirical Context
  10. Model and Implementation
  11. Parameters of Interest and Associated Results
  12. Marginal Effects
  13. Optimal Personalized Offers
  14. Theoretical Results
  15. Bounds for Structural Deep Learning
  16. ...and 19 more sections

Key Result

Theorem 1

Let $\bm{w}_i$, $i=1,\ldots,n$, be a random sample that obeys Assumptions asmpt:loss and asmpt:dgp dnn. Define $\bm{\widehat{\theta}}$ as the estimator found by solving eqn:first stage, where the class $\mathcal{F}_{\textsc{dnn}}$ is a feedforward, fully connected network with ReLU activation struct and for $n$ large enough, with probability $1-\exp\{-n^{\frac{d_c}{p+d_c}} \log^8 n\}$, for $k=1,\

Figures (7)

  • Figure 1: Structural deep learning. A schematic depiction of the deep neural network architecture for estimating the parameter functions $\boldsymbol{\theta}(\bm{x})$ in the structural economic model $\ell(\bm{Y}, \bm{T}, \boldsymbol{\theta}(\bm{X}))$ defined in Eqn. \ref{['eqn:first stage']}.
  • Figure 2: Structural modeling and machine learning for demand functions. Panels (a), (b), and (c) show estimated demand functions using random forests, neural networks, and a structural binary choice model, respectively, using the data of Bertrand-etal2010_QJE. Panels (d) and (e) show the extrapolated demand functions and (f) the implied revenue functions.
  • Figure 3: Marginal Effect of Advertising Content
  • Figure 4: Optimal Personalized Interest Rate Offers
  • Figure 5: Expected Profits from Personalized Interest Rate Offers
  • ...and 2 more figures

Theorems & Definitions (14)

  • Remark 3.1: Two Step GMM
  • Remark 3.2
  • Remark 3.3: Notes on $\boldsymbol{\Lambda}(\bm{x})$
  • Remark 3.4: Auto-DML
  • Remark 4.1: Implementation Uncertainty
  • Theorem 1
  • Theorem 2
  • Remark 5.1: Efficiency
  • Remark 5.2: High Dimensional Parametric Approach
  • Remark 5.3: Other Uses of Influence Functions
  • ...and 4 more