Indeterminacy in Affective Computing: Considering Meaning and Context in Data Collection Practices

TL;DR

This paper addresses the problem that Automatic Affect Prediction (AAP) models rely on labeled data whose meanings are produced by human Affective Interpretation Processes (AIPs) and are inherently indeterminate. It proposes a conceptual framework that formalizes AIPs, Affective Meaning, and four Qualities of Indeterminacy (Subjectivity, Uncertainty, Ambiguity, Vagueness) and introduces Context Aspects and Context Configurations to explain how interpretation is shaped by context. The authors distinguish Phenomenon Configuration (real-world conditions) from Measurement Configuration (data-collection setups) and illustrate how measurement designs can distort or exaggerate QIs, thereby affecting model predictions. The core contribution is a two-step program: (i) identify and measure relevant QIs, and (ii) systematically document contextual variables that influence interpretation, aiming to improve data collection practices and the real-world reliability of AAP systems.

Abstract

Automatic Affect Prediction (AAP) uses computational analysis of input data such as text, speech, images, and physiological signals to predict various affective phenomena (e.g., emotions or moods). These models are typically constructed using supervised machine-learning algorithms, which rely heavily on labeled training datasets. In this position paper, we posit that all AAP training data are derived from human Affective Interpretation Processes, resulting in a form of Affective Meaning. Research on human affect indicates a form of complexity that is fundamental to such meaning: it can possess what we refer to here broadly as Qualities of Indeterminacy (QIs) - encompassing Subjectivity (meaning depends on who is interpreting), Uncertainty (lack of confidence regarding meanings' correctness), Ambiguity (meaning contains mutually exclusive concepts) and Vagueness (meaning is situated at different levels in a nested hierarchy). Failing to appropriately consider QIs leads to results incapable of meaningful and reliable predictions. Based on this premise, we argue that a crucial step in adequately addressing indeterminacy in AAP is the development of data collection practices for modeling corpora that involve the systematic consideration of 1) a relevant set of QIs and 2) context for the associated interpretation processes. To this end, we are 1) outlining a conceptual model of AIPs and the QIs associated with the meaning these produce and a conceptual structure of relevant context, supporting understanding of its role. Finally, we use our framework for 2) discussing examples of context-sensitivity-related challenges for addressing QIs in data collection setups. We believe our efforts can stimulate a structured discussion of both the role of aspects of indeterminacy and context in research on AAP, informing the development of better practices for data collection and analysis.

Figures (1)

• Figure 1: Conceptual Model of Affect Interpretation Processes