Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Emergent, not Immanent: A Baradian Reading of Explainable AI

Fabio Morreale, Joan Serrà, Yuki Mistufuji

TL;DR

The paper critiques mainstream XAI for treating explanations as intrinsic to models, arguing instead that interpretability arises from situated intra-actions among models, interpreters, data, and contexts. Grounded in Barad's agential realism and diffraction, it reframes explanations as material-discursive performances that are emergent rather than immanent, and analyzes a broad set of XAI methods through this lens. It identifies limitations of many methods (e.g., assuming pre-existing explananda, faithfulness concerns) and champions a diffractive approach that surfaces multiple, context-dependent interpretations while emphasizing responsibility and accountability within entanglements. Finally, it outlines diffractive design directions for interfaces that foreground plurality, situatedness, and ongoing engagement, illustrated via a speculative text-to-music case study.

Abstract

Explainable AI (XAI) is frequently positioned as a technical problem of revealing the inner workings of an AI model. This position is affected by unexamined onto-epistemological assumptions: meaning is treated as immanent to the model, the explainer is positioned outside the system, and a causal structure is presumed recoverable through computational techniques. In this paper, we draw on Barad's agential realism to develop an alternative onto-epistemology of XAI. We propose that interpretations are material-discursive performances that emerge from situated entanglements of the AI model with humans, context, and the interpretative apparatus. To develop this position, we read a comprehensive set of XAI methods through agential realism and reveal the assumptions and limitations that underpin several of these methods. We then articulate the framework's ethical dimension and propose design directions for XAI interfaces that support emergent interpretation, using a speculative text-to-music interface as a case study.

Emergent, not Immanent: A Baradian Reading of Explainable AI

TL;DR

The paper critiques mainstream XAI for treating explanations as intrinsic to models, arguing instead that interpretability arises from situated intra-actions among models, interpreters, data, and contexts. Grounded in Barad's agential realism and diffraction, it reframes explanations as material-discursive performances that are emergent rather than immanent, and analyzes a broad set of XAI methods through this lens. It identifies limitations of many methods (e.g., assuming pre-existing explananda, faithfulness concerns) and champions a diffractive approach that surfaces multiple, context-dependent interpretations while emphasizing responsibility and accountability within entanglements. Finally, it outlines diffractive design directions for interfaces that foreground plurality, situatedness, and ongoing engagement, illustrated via a speculative text-to-music case study.

Abstract

Explainable AI (XAI) is frequently positioned as a technical problem of revealing the inner workings of an AI model. This position is affected by unexamined onto-epistemological assumptions: meaning is treated as immanent to the model, the explainer is positioned outside the system, and a causal structure is presumed recoverable through computational techniques. In this paper, we draw on Barad's agential realism to develop an alternative onto-epistemology of XAI. We propose that interpretations are material-discursive performances that emerge from situated entanglements of the AI model with humans, context, and the interpretative apparatus. To develop this position, we read a comprehensive set of XAI methods through agential realism and reveal the assumptions and limitations that underpin several of these methods. We then articulate the framework's ethical dimension and propose design directions for XAI interfaces that support emergent interpretation, using a speculative text-to-music interface as a case study.
Paper Structure (31 sections, 2 figures, 1 table)

This paper contains 31 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Positivism and Representationalism
  4. Optics of Knowledge Production
  5. Agential Realism
  6. Feminist Epistemology and XAI
  7. Agential Realism and Diffraction in HCI
  8. XAI Methods
  9. Feature Attribution
  10. Sample-based
  11. Probing-based
  12. Mechanistic Interpretability
  13. Interactive XAI
  14. A Baradian Analysis of XAI methods
  15. Refractive and Reflective Reading of XAI
  16. ...and 16 more sections

Figures (2)

  • Figure 1: A few of the features grouped into the "Michael Jordan" supernode in ameisen_circuit_2025.
  • Figure 2: A speculative interface designed to interpret text-to-music generations. It displays multiple situated entanglements annotated by different users.