Characterizing Information Seeking Processes with Multiple Physiological Signals

TL;DR

This study addresses how cognitive load and affect during interactive information seeking can be quantified using multi-modal physiological signals. It combines four information-seeking stages—$IN$, $QF$, $QS$, and $RJ$—with text and audio modalities in a controlled lab, collecting $EDA$, $PPG$, $EEG$, and pupillometry to derive indices such as $TAR$, $BAR$, $FAA$, $SCL$, $HRV$, and $RPD$, mapped to $H_{cog}$, $H_{aro}$, and $H_{val}$. The results show higher cognitive load at $IN$ relative to $QF$, greater load during $QS$ than $QF$ or $RJ$, and heightened affective responses during $RJ$, while valence remains largely neutral. These findings provide a quantitative baseline for modeling user states in information seeking and pave the way for real-time, physiology-informed adaptive information systems, including future conversational search with large language models. The work establishes methodological groundwork and motivates extending physiology-based IIR analyses to more complex, interactive contexts.

Abstract

Information access systems are getting complex, and our understanding of user behavior during information seeking processes is mainly drawn from qualitative methods, such as observational studies or surveys. Leveraging the advances in sensing technologies, our study aims to characterize user behaviors with physiological signals, particularly in relation to cognitive load, affective arousal, and valence. We conduct a controlled lab study with 26 participants, and collect data including Electrodermal Activities, Photoplethysmogram, Electroencephalogram, and Pupillary Responses. This study examines informational search with four stages: the realization of Information Need (IN), Query Formulation (QF), Query Submission (QS), and Relevance Judgment (RJ). We also include different interaction modalities to represent modern systems, e.g., QS by text-typing or verbalizing, and RJ with text or audio information. We analyze the physiological signals across these stages and report outcomes of pairwise non-parametric repeated-measure statistical tests. The results show that participants experience significantly higher cognitive loads at IN with a subtle increase in alertness, while QF requires higher attention. QS involves demanding cognitive loads than QF. Affective responses are more pronounced at RJ than QS or IN, suggesting greater interest and engagement as knowledge gaps are resolved. To the best of our knowledge, this is the first study that explores user behaviors in a search process employing a more nuanced quantitative analysis of physiological signals. Our findings offer valuable insights into user behavior and emotional responses in information seeking processes. We believe our proposed methodology can inform the characterization of more complex processes, such as conversational information seeking.

Figures (5)

• Figure 1: The flow chart presents how the information is transformed through search stages, 1) the realization of Information Need ( IN), 2) Query Formulation ( QF), 3) Query Submission ( QS) and 4) Relevance Judgment ( RJ), in information seeking process, based on the combination and unification of the previous models.
• Figure 2: Experimental Procedure. IN: the realization of Information Need.
• Figure 3: Experiment setup (left), and the EEG electrode locations (right). The filled circles indicate the electrodes used to compute the indexes.
• Figure 4: Distribution of indexes for measuring cognitive load across all participants. The values of one participant are aggregated into one data point.
• Figure 5: Distribution of indexes for measuring affective arousal (\ref{['subfig:BAR']}, \ref{['subfig:SCL']}, \ref{['subfig:HRV']}) and valence (\ref{['subfig:FAA']}, \ref{['subfig:FAA_components']}). Error bars indicate standard error.