Deep Learning Opacity in Scientific Discovery

TL;DR

The paper tackles the problem of deep learning opacity in scientific practice by distinguishing how outputs are used across discovery and justification contexts. It argues that opacity need not impede genuine scientific progress when DL serves as a tool for discovery rather than as a final, justificatory claim, and it supports this with two case studies: Case 1 in knot theory, where a DL-guided intuition leads to a rigorous theorem $|2 \sigma(\mathcal{K})-\operatorname{slope}(\mathcal{K})| \le c\,\operatorname{vol}(\mathcal{K})\,\operatorname{inj}(\mathcal{K})^{-3}$, and Case 2 in earthquake geophysics, where a DL predictor with $AUC = 0.85$ helps identify relevant physical properties that substantially improve theory. Together, these cases illustrate that epistemic opacity can be epistemically irrelevant for justified discoveries, provided findings are ultimately evaluated by standard disciplinary justification. This reframes the debate by showing epistemic transparency is not universally required for DL-assisted science, though it remains crucial for outputs treated as knowledge claims. The work underscores a nuanced view of AI’s role in science, with practical implications for how philosophers assess the epistemic value of opaque models and how scientists integrate DL into discovery workflows.

Abstract

Philosophers have recently focused on critical, epistemological challenges that arise from the opacity of deep neural networks. One might conclude from this literature that doing good science with opaque models is exceptionally challenging, if not impossible. Yet, this is hard to square with the recent boom in optimism for AI in science alongside a flood of recent scientific breakthroughs driven by AI methods. In this paper, I argue that the disconnect between philosophical pessimism and scientific optimism is driven by a failure to examine how AI is actually used in science. I show that, in order to understand the epistemic justification for AI-powered breakthroughs, philosophers must examine the role played by deep learning as part of a wider process of discovery. The philosophical distinction between the 'context of discovery' and the 'context of justification' is helpful in this regard. I demonstrate the importance of attending to this distinction with two cases drawn from the scientific literature, and show that epistemic opacity need not diminish AI's capacity to lead scientists to significant and justifiable breakthroughs.

Figures (1)

• Figure 1: The confinement of epistemically opaque, neural network outputs to the context of discovery. a) posit or assume the existence of some theory $\exists{f}$ that connects two phenomena $\mathcal{X}$ and $\mathcal{Y}$; b) generate a dataset $\mathcal{D}$ that represents the assumed connection; c) train a deep learning model to learn a function $\hat{f}$ that approximates the posited theory; d) examine the behavior of $\hat{f}$; e) iteratively evaluate (b)-(d), formulate, and refine conjectures or hypotheses $f_i^*$ that connect $\mathcal{X}$ and $\mathcal{Y}$. Justify $f^*$ by means distinct from those used to produce it.