What to Make Sense of in the Era of LLM? A Perspective from the Structure and Efforts in Sensemaking

TL;DR

This study explores how LLMs (i.e., GPT-4) can assist in a complex sensemaking task of deciphering fictional terrorist plots, and explores two different approaches for leveraging GPT-4's capabilities: a holistic sensemaking process and a step-by-step approach.

Abstract

Sensemaking tasks often entail navigating through complex, ambiguous data to construct coherent insights. Prior work has shown that crowds can effectively distribute cognitive load, pooling diverse perspectives to enhance analytical depth. Recent advancements in LLMs have further expanded the toolkit for sensemaking, offering scalable data processing, complex pattern recognition, and the ability to infer and propose meaningful hypotheses. In this study, we explore how LLMs (i.e., GPT-4) can assist in a complex sensemaking task of deciphering fictional terrorist plots. We explore two different approaches for leveraging GPT-4's capabilities: a holistic sensemaking process and a step-by-step approach. Our preliminary investigations open the doors for future research into optimizing human-AI collaborative workflows, aiming to harness the complementary strengths of both for more effective sensemaking in complex scenarios.

Figures (2)

• Figure 1: Efficacy in Fact Extraction (shadowed parts are facts that are not comprehensively covered).
• Figure 2: Efficacy in Hypotheses Development (shadowed parts are hypotheses that are nto comprehensively covered)