Understanding Trust Toward Human versus AI-generated Health Information through Behavioral and Physiological Sensing

TL;DR

The paper investigates how people trust online health information from human professionals versus AI-generated sources, examining the role of transparency labels. Through a mixed-methods online survey and a lab study with eye-tracking and physiological sensing, it shows that AI-generated content is generally trusted more than human-generated content, while information labeled as human is trusted more than AI-labeled content; trust effects are consistent across information types. Behavioral and physiological data reveal distinct patterns by source and label, and ML analyses demonstrate that gaze and autonomic signals can predict trust and classify information source with meaningful accuracy. These findings illuminate how transparency cues shape trust and suggest avenues for building trust-aware, disclosure-responsive health information interfaces and adaptive AI systems. The work underscores the importance of labeling design and the potential of multimodal sensing to verify trust perceptions in AI-assisted health information contexts.

Abstract

As AI-generated health information proliferates online and becomes increasingly indistinguishable from human-sourced information, it becomes critical to understand how people trust and label such content, especially when the information is inaccurate. We conducted two complementary studies: (1) a mixed-methods survey (N=142) employing a 2 (source: Human vs. LLM) $\times$ 2 (label: Human vs. AI) $\times$ 3 (type: General, Symptom, Treatment) design, and (2) a within-subjects lab study (N=40) incorporating eye-tracking and physiological sensing (ECG, EDA, skin temperature). Participants were presented with health information varying by source-label combinations and asked to rate their trust, while their gaze behavior and physiological signals were recorded. We found that LLM-generated information was trusted more than human-generated content, whereas information labeled as human was trusted more than that labeled as AI. Trust remained consistent across information types. Eye-tracking and physiological responses varied significantly by source and label. Machine learning models trained on these behavioral and physiological features predicted binary self-reported trust levels with 73% accuracy and information source with 65% accuracy. Our findings demonstrate that adding transparency labels to online health information modulates trust. Behavioral and physiological features show potential to verify trust perceptions and indicate if additional transparency is needed.

Figures (13)

• Figure 1: Visual summary of the studies in this paper. (a) Study 1: Mixed-methods crowdsourcing survey study to measure perceived trust; (b) Study 2: Within-subjects lab study to measure perceived trust, as well as behavioral and physiological responses.
• Figure 2: Example reading task from the survey, showing a Q&A pair with its assigned disclosed source label. Each participant read six Q&A pairs: three labeled as from "AI" (left) and three labeled as from "Human Professionals" (middle). After each reading, participants rated their trust using the scale shown on the right.
• Figure 3: Left: Perceived trust score in information by sources regardless of labels, and by labels regardless of source from the three-way mixed ANOVA without correction. Right: Post hoc pairwise comparisons on perceived trust score based on different source and label conditions using t-test with False Discovery Rate (FDR) correction. Each plot shows the score density (width), with the red dot indicating the mean, the black line as the median, and thick bars representing the interquartile range (IQR). Horizontal lines indicate significance (**$p$<.01, *$p$<.05, "ns": no significance).
• Figure 4: Pearson correlation with Bonferroni correction among the key variables in the online survey.(**$p$<.01, *$p$<.05) Note: "HumLabel": information with human label regardless of the actual source. "AILabel": information with AI label regardless of the actual source.
• Figure 5: Top: An example of text stimulus displayed on the monitor. Bottom: Heatmap of the gaze points on stimuli. Three AOIs are predefined: AOI-1 is the area for presenting disclosed label; AOI-2 is the area for presenting health information; AOI-3 is the area to rate the perceived trust in health information.