Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploratory Causal Inference in SAEnce

Tommaso Mencattini, Riccardo Cadei, Francesco Locatello

TL;DR

The paper tackles the challenge of discovering causal effects in randomized trials when outcomes are high-dimensional and indirect. It introduces Neural Effect Search (NES), a recursive, stratified testing framework that operates on neural representations learned from foundation models and sparsified by autoencoders to identify principal directions of causal impact. By formalizing an empiricist–exploratory approach and proving consistency under standard RCT assumptions, the authors demonstrate that NES mitigates entanglement leakage and avoids the multiplicity-driven significance collapse that plagues conventional tests. Empirical validation on semi-synthetic CelebA benchmarks and a real-world ecological RCT shows NES can recover known effects (e.g., grooming) and reveal interpretable, domain-relevant signals, offering a scalable, data-driven rescue mechanism for hypothesis generation. Overall, this work advances AI-assisted scientific discovery by enabling large-scale, interpretable causal inference from rich observational data while complementing traditional rationalist analyses.

Abstract

Randomized Controlled Trials are one of the pillars of science; nevertheless, they rely on hand-crafted hypotheses and expensive analysis. Such constraints prevent causal effect estimation at scale, potentially anchoring on popular yet incomplete hypotheses. We propose to discover the unknown effects of a treatment directly from data. For this, we turn unstructured data from a trial into meaningful representations via pretrained foundation models and interpret them via a sparse autoencoder. However, discovering significant causal effects at the neural level is not trivial due to multiple-testing issues and effects entanglement. To address these challenges, we introduce Neural Effect Search, a novel recursive procedure solving both issues by progressive stratification. After assessing the robustness of our algorithm on semi-synthetic experiments, we showcase, in the context of experimental ecology, the first successful unsupervised causal effect identification on a real-world scientific trial.

Exploratory Causal Inference in SAEnce

TL;DR

The paper tackles the challenge of discovering causal effects in randomized trials when outcomes are high-dimensional and indirect. It introduces Neural Effect Search (NES), a recursive, stratified testing framework that operates on neural representations learned from foundation models and sparsified by autoencoders to identify principal directions of causal impact. By formalizing an empiricist–exploratory approach and proving consistency under standard RCT assumptions, the authors demonstrate that NES mitigates entanglement leakage and avoids the multiplicity-driven significance collapse that plagues conventional tests. Empirical validation on semi-synthetic CelebA benchmarks and a real-world ecological RCT shows NES can recover known effects (e.g., grooming) and reveal interpretable, domain-relevant signals, offering a scalable, data-driven rescue mechanism for hypothesis generation. Overall, this work advances AI-assisted scientific discovery by enabling large-scale, interpretable causal inference from rich observational data while complementing traditional rationalist analyses.

Abstract

Randomized Controlled Trials are one of the pillars of science; nevertheless, they rely on hand-crafted hypotheses and expensive analysis. Such constraints prevent causal effect estimation at scale, potentially anchoring on popular yet incomplete hypotheses. We propose to discover the unknown effects of a treatment directly from data. For this, we turn unstructured data from a trial into meaningful representations via pretrained foundation models and interpret them via a sparse autoencoder. However, discovering significant causal effects at the neural level is not trivial due to multiple-testing issues and effects entanglement. To address these challenges, we introduce Neural Effect Search, a novel recursive procedure solving both issues by progressive stratification. After assessing the robustness of our algorithm on semi-synthetic experiments, we showcase, in the context of experimental ecology, the first successful unsupervised causal effect identification on a real-world scientific trial.
Paper Structure (34 sections, 10 theorems, 53 equations, 12 figures, 2 tables, 2 algorithms)

This paper contains 34 sections, 10 theorems, 53 equations, 12 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Treatment Effect Estimation in Randomized Controlled Trials
  3. Exploratory Causal Inference via Neural Representations
  4. Neural Effect Search
  5. Related Works
  6. Experiments
  7. Semi-Synthetic Benchmark
  8. Real-World Randomized Trial from Experimental Ecology
  9. Conclusion
  10. Proofs
  11. Significance level collapse with sample size (Theorem \ref{['thm:powercollapse_n']})
  12. Significance collapse with effect magnitude (Corollary \ref{['thm:powercollapse_tau']})
  13. Consistency of Neural Effect Search (Theorem \ref{['thm:nes_consistency']})
  14. Residualization (optional).
  15. Neural Effect Test
  16. ...and 19 more sections

Key Result

Theorem 3.1

Suppose at least $\rho_\varepsilon m$ neurons have nonzero effect $|\tau_j|\ge \varepsilon>0$. Via multiple testing, regardless of the Bonferroni adjustment, and the number of rejections converges to $\rho_\varepsilon m$ in probability.

Figures (12)

  • Figure 1: Pipeline for Exploratory Causal Inference: (i) Collect data from a Randomized Controlled Trial, (ii) Extract representations via a Foundation Model and Sparse Autoencoder, (iii) Apply Neural Effect Search, and (iv) domain experts interpret the causal findings.
  • Figure 2: Exemplary graphical models for randomized controlled trials (i.e., no edge from $W$ to $T$). In Causal Inference (left), both $T$ and $Y$ are observed, and $W$ does not influence $T$ as we are assuming a randomized controlled trial. In Prediction-Powered Causal Inference (center), $Y$ is not observed directly but is known and can be partially labeled. The missing $Y$ is predicted by a neural network from high-dimensional $X$ that is trained either on the same trials if labels are available cadei2024smoke or on other trials with the same label space cadei2025causal. In Exploratory Causal Inference (right), $Y$ is unknown and unobserved and is discovered by a neural network from high-dimensional $X$ in a purely data-driven way.
  • Figure 3: The Paradox of Exploratory Causal Inference: Increasing the power of the test, the effect on any outcome-entangled code becomes significant, regardless of its main interpretation.
  • Figure 4: Semi-synthetic benchmark. Precision, Recall, and IoU of different testing procedures across sample size $N$ (top) and effect size $\tau$ (bottom). NES consistently achieves the best trade-off, avoiding the significance collapse of standard corrections.
  • Figure 5: Exploratory Causal Inference for Experimental Ecology. Without any knowledge of the behaviors of interest, our procedure retrieves two significant treatment effects, i.e., grooming and background, in agreement with previous literature.
  • ...and 7 more figures

Theorems & Definitions (19)

  • Definition 3.1: Leakage set and index
  • Theorem 3.1: Significance level collapse with sample size
  • Theorem 3.2: Significance collapse with effect magnitude
  • proof : Proof sketch
  • Theorem 4.1: Consistency of Neural Effect Search
  • proof : Proof Sketch
  • Theorem A.1: Significance level collapse with sample size
  • proof
  • Corollary A.1: Significance collapse with effect magnitude
  • proof
  • ...and 9 more