Decision-Making Amid Information-Based Threats in Sociotechnical Systems: A Review

TL;DR

The paper addresses how information-based threats shape decision-making in sociotechnical systems, including human-AI interactions, by integrating findings from cognitive processing and threat literature. It outlines shared cognitive, affective, and social mediators (e.g., illusory truth, credibility, affect, time pressure) and surveys a range of frameworks (Bayesian, information-processing, naturalistic decision-making, and team models) while identifying gaps for integration. The key contribution is a call for an integrative representational framework that captures how threats influence behavior across individuals and teams, with an emphasis on ecological validity and practical safeguards for human decision-makers and human-AI systems. This integrative approach aims to advance cognitive security and improve resilience and alignment in sociotechnical environments facing evolving information-based threats.

Abstract

Technological systems increasingly mediate human information exchange, spanning interactions among humans as well as between humans and artificial agents. The unprecedented scale and reliance on information disseminated through these systems substantially expand the scope of information-based influence that can both enable and undermine sound decision-making. Consequently, understanding and protecting decision-making today faces growing challenges, as individuals and organizations must navigate evolving opportunities and information-based threats across varied domains and information environments. While these risks are widely recognized, research remains fragmented: work evaluating information-based threat phenomena has progressed largely in isolation from foundational studies of human information processing. In this review, we synthesize insights from both domains to identify shared cognitive mechanisms that mediate vulnerability to information-based threats and shape behavioral outcomes. Finally, we outline directions for future research aimed at integrating these perspectives, emphasizing the importance of such integration for mitigating human vulnerabilities and aligning human-machine representations.

Figures (1)

• Figure 1: An overview of this paper's literature review, synthesizing fields across human psychology to better understand decision-making in sociotechnical systems in modern contexts. In the information processing domain (connected by solid, teal lines), much empirical and theoretical work has been conducted, spanning numerous sub-fields (indicated by the smaller circles: i.e., JDM, HFE, NDM, and I-O Psychology). In the information-based threats domain (red coloring), a primary focus over the last decade, a new field has emerged with complementary findings and theories. The integration of these domains (the central circle and focus of this review) provides an improved understanding of human cognition and behavior broadly. Where this effort fits in with the larger goal of aligning (and securing) artificial intelligence systems and neuroscience with human cognitive mechanisms and behavior is also depicted (three white circles, which all influence each other; represented with bidirectional arrows; reviewed in sucholutsky_getting_2024).