The Representational Status of Deep Learning Models

TL;DR

The paper tackles whether deep learning models encode locally decomposable, semantically meaningful representations of their targets. It argues for a relational conception of representation, $representation_{rel}$, but shows that DL models typically exhibit global, holistic relational properties rather than stable, local content. Through analysis of scientific-model literature, composition vs. distribution arguments, and the role of opacity, it contends that DL representations are best viewed as holistic $representation_{rel}$ with limited grounds for fine-grained semantic decomposition. This reframing carries epistemic and practical implications for XAI and suggests focusing on global relational properties when leveraging DLs for scientific inquiry, while remaining cautious about world-directed interpretations.

Abstract

This paper aims to clarify the representational status of Deep Learning Models (DLMs). While commonly referred to as 'representations', what this entails is ambiguous due to a conflation of functional and relational conceptions of representation. This paper argues that while DLMs represent their targets in a relational sense, in general, we have no good reason to believe that DLMs encode locally semantically decomposable representations of their targets. That is, the representational capacity these models have is largely global, rather than decomposable into stable, local subrepresentations. This result has immediate implications for explainable AI (XAI) and directs attention toward exploring the global relational nature of deep learning representations and their relationship both to models more generally to understand their potential role in future scientific inquiry.

Figures (1)

• Figure 1: Degree of idealization and degree of semantic decomposability each form a separate dimension of model-building. Some models, like a scale model, are highly decomposable and minimally idealized. Others, like the Ising model, are minimally decomposable and highly idealized. Deep learning models are minimally decomposable, but, for any given model, it is not possible to determine its degree of idealization due to opacity.