Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NESTER: An Adaptive Neurosymbolic Method for Causal Effect Estimation

Abbavaram Gowtham Reddy, Vineeth N Balasubramanian

TL;DR

NESTER tackles causal effect estimation from observational data by fusing neurosymbolic program synthesis with a Domain-Specific Language tailored to causal inference. It synthesizes differentiable programs via an A*-driven search, using primitives that encode inductive biases from TARNet, CFR, Dragonnet, and related methods, and learns both the program structure $\mathcal{P}$ and parameters $\theta$. The approach comes with theoretical guarantees, including an $\epsilon$-admissible heuristic and universal approximation results, and demonstrates state-of-the-art performance on semi-synthetic IHDP and Twins datasets as well as the real Jobs dataset. The framework is capable of reproducing existing architectures as special cases and offers interpretability through the synthesized primitive composition, with practical implications for adaptable, data-driven causal effect estimation. Future work includes expanding the DSL, mitigating runtime and scalability concerns, and relaxing latent-confounding assumptions.

Abstract

Causal effect estimation from observational data is a central problem in causal inference. Methods based on potential outcomes framework solve this problem by exploiting inductive biases and heuristics from causal inference. Each of these methods addresses a specific aspect of causal effect estimation, such as controlling propensity score, enforcing randomization, etc., by designing neural network (NN) architectures and regularizers. In this paper, we propose an adaptive method called Neurosymbolic Causal Effect Estimator (NESTER), a generalized method for causal effect estimation. NESTER integrates the ideas used in existing methods based on multi-head NNs for causal effect estimation into one framework. We design a Domain Specific Language (DSL) tailored for causal effect estimation based on causal inductive biases used in literature. We conduct a theoretical analysis to investigate NESTER's efficacy in estimating causal effects. Our comprehensive empirical results show that NESTER performs better than state-of-the-art methods on benchmark datasets.

NESTER: An Adaptive Neurosymbolic Method for Causal Effect Estimation

TL;DR

NESTER tackles causal effect estimation from observational data by fusing neurosymbolic program synthesis with a Domain-Specific Language tailored to causal inference. It synthesizes differentiable programs via an A*-driven search, using primitives that encode inductive biases from TARNet, CFR, Dragonnet, and related methods, and learns both the program structure and parameters . The approach comes with theoretical guarantees, including an -admissible heuristic and universal approximation results, and demonstrates state-of-the-art performance on semi-synthetic IHDP and Twins datasets as well as the real Jobs dataset. The framework is capable of reproducing existing architectures as special cases and offers interpretability through the synthesized primitive composition, with practical implications for adaptable, data-driven causal effect estimation. Future work includes expanding the DSL, mitigating runtime and scalability concerns, and relaxing latent-confounding assumptions.

Abstract

Causal effect estimation from observational data is a central problem in causal inference. Methods based on potential outcomes framework solve this problem by exploiting inductive biases and heuristics from causal inference. Each of these methods addresses a specific aspect of causal effect estimation, such as controlling propensity score, enforcing randomization, etc., by designing neural network (NN) architectures and regularizers. In this paper, we propose an adaptive method called Neurosymbolic Causal Effect Estimator (NESTER), a generalized method for causal effect estimation. NESTER integrates the ideas used in existing methods based on multi-head NNs for causal effect estimation into one framework. We design a Domain Specific Language (DSL) tailored for causal effect estimation based on causal inductive biases used in literature. We conduct a theoretical analysis to investigate NESTER's efficacy in estimating causal effects. Our comprehensive empirical results show that NESTER performs better than state-of-the-art methods on benchmark datasets.
Paper Structure (20 sections, 6 theorems, 35 equations, 7 figures, 6 tables, 1 algorithm)

This paper contains 20 sections, 6 theorems, 35 equations, 7 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background and Problem Formulation
  4. NESTER: Methodology
  5. DSL for Causal Effect Estimation
  6. DSL Encodes Causal Inductive Biases
  7. NESTER Algorithm
  8. NESTER: Analysis
  9. Experiments and Results
  10. Limitations and Conclusions
  11. Proofs of Propositions
  12. Algorithm
  13. Experimental Setup
  14. Details on Datasets
  15. Additional Details on Evaluation Metrics
  16. ...and 5 more sections

Key Result

Lemma 5.1

(Neural Admissible Relaxations near) In an informed search algorithm $\mathcal{A}$, given an internal node $u_i$ and a leaf node $u_l$, let the cost of the leaf edge $(u_i, u_l)$ be $s(r)+\zeta(\mathcal{P}, \theta^{*})$, where $\theta^{*} = \mathop{\mathrm{arg\,min}}\limits_{\theta} \zeta(\mathcal{P

Figures (7)

  • Figure 1: (a) TARNet architecture. $\mathbf{X}$ is feature vector, $t$ is treatment, $\hat{Y}_1, \hat{Y}_0$ are estimated potential outcomes and $\phi$ is learned representation at the end of shared layers. (b) Program $\mathcal{P}_T$ synthesized by NESTER using our Domain-Specific Language (DSL) (Tab \ref{['tab:dsl_causal_inference']}) that is functionally similar to TARNet. Colors are used to show the equivalence between the components of TARNet and $\mathcal{P}_T$.
  • Figure 2: Example program tree generated using DSL in Tab \ref{['tab:dsl_causal_inference']}. Structural costs are shown in red color (e.g., $s(r)=0.2$ for the rule $r: \rho \rightarrow \alpha_1+\alpha_2$). $h$ is the heuristic value, and $f$ is the sum of structural cost and heuristic value. The path from the root node to a leaf node returned by $A^*$ algorithm is shown in blue color.
  • Figure A1: (a) CFR architecture (b) Program $\mathcal{P}_C$ synthesized by NESTER using our DSL (Tab \ref{['tab:dsl_causal_inference']}) that is functionally similar to CFR. Colors are used to show the equivalence between the components of CFR and $\mathcal{P}_C$.
  • Figure A2: (a) Dragonnet architecture (b) Program $\mathcal{P}_D$ synthesized by NESTER using our DSL (Tab \ref{['tab:dsl_causal_inference']}) that is functionally similar to Dragonnet. Colors are used to show the equivalence between the components of Draonnet and $\mathcal{P}_D$.
  • Figure A3: Feature count vs $\epsilon_{ACE}$
  • ...and 2 more figures

Theorems & Definitions (14)

  • Definition 3.1
  • Definition 3.2
  • Definition 3.3
  • Definition 3.4
  • Definition 5.1
  • Lemma 5.1
  • Proposition 5.1
  • Proposition 5.2
  • Lemma A.1
  • proof
  • ...and 4 more