Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OSINT Research Studios: A Flexible Crowdsourcing Framework to Scale Up Open Source Intelligence Investigations

Anirban Mukhopadhyay, Sukrit Venkatagiri, Kurt Luther

TL;DR

The paper presents OSINT Research Studios (ORS), a design-based framework that trains novices to assist expert-led OSINT investigations through five macrotasks (discovery, source analysis, image analysis, verification, geolocation). Using a semester-long OSINT lab course and a hybrid expert-crowd setup, ORS demonstrates scalable, ethical collaboration that improves speed, quality, and breadth of leads while maintaining safety and transparency. The findings show the trained crowd can contribute meaningfully, with experts maintaining control and providing targeted feedback to steer investigations. The work discusses broader implications for expert-crowd collaboration, domain expansion, and future automation to further scale OSINT investigations across disciplines.

Abstract

Open Source Intelligence (OSINT) investigations, which rely entirely on publicly available data such as social media, play an increasingly important role in solving crimes and holding governments accountable. The growing volume of data and complex nature of tasks, however, means there is a pressing need to scale and speed up OSINT investigations. Expert-led crowdsourcing approaches show promise but tend to either focus on narrow tasks or domains or require resource-intense, long-term relationships between expert investigators and crowds. We address this gap by providing a flexible framework that enables investigators across domains to enlist crowdsourced support for the discovery and verification of OSINT. We use a design-based research (DBR) approach to develop OSINT Research Studios (ORS), a sociotechnical system in which novice crowds are trained to support professional investigators with complex OSINT investigations. Through our qualitative evaluation, we found that ORS facilitates ethical and effective OSINT investigations across multiple domains. We also discuss broader implications of expert-crowd collaboration and opportunities for future work.

OSINT Research Studios: A Flexible Crowdsourcing Framework to Scale Up Open Source Intelligence Investigations

TL;DR

The paper presents OSINT Research Studios (ORS), a design-based framework that trains novices to assist expert-led OSINT investigations through five macrotasks (discovery, source analysis, image analysis, verification, geolocation). Using a semester-long OSINT lab course and a hybrid expert-crowd setup, ORS demonstrates scalable, ethical collaboration that improves speed, quality, and breadth of leads while maintaining safety and transparency. The findings show the trained crowd can contribute meaningfully, with experts maintaining control and providing targeted feedback to steer investigations. The work discusses broader implications for expert-crowd collaboration, domain expansion, and future automation to further scale OSINT investigations across disciplines.

Abstract

Open Source Intelligence (OSINT) investigations, which rely entirely on publicly available data such as social media, play an increasingly important role in solving crimes and holding governments accountable. The growing volume of data and complex nature of tasks, however, means there is a pressing need to scale and speed up OSINT investigations. Expert-led crowdsourcing approaches show promise but tend to either focus on narrow tasks or domains or require resource-intense, long-term relationships between expert investigators and crowds. We address this gap by providing a flexible framework that enables investigators across domains to enlist crowdsourced support for the discovery and verification of OSINT. We use a design-based research (DBR) approach to develop OSINT Research Studios (ORS), a sociotechnical system in which novice crowds are trained to support professional investigators with complex OSINT investigations. Through our qualitative evaluation, we found that ORS facilitates ethical and effective OSINT investigations across multiple domains. We also discuss broader implications of expert-crowd collaboration and opportunities for future work.
Paper Structure (102 sections, 2 figures, 3 tables)

This paper contains 102 sections, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. OSINT Investigations
  4. Collaborative Sensemaking and Crowdsourced Investigations
  5. Collaborative sensemaking
  6. Crowdsourced investigations
  7. Decomposition and Training for Crowdsourcing Complex Work
  8. Design Challenges and Opportunities for Crowdsourcing Complex OSINT Investigations
  9. Delegation of Complex OSINT Tasks
  10. Discovery
  11. Challenges:
  12. Design Goals:
  13. Verification
  14. Challenges:
  15. Design Goals:
  16. ...and 87 more sections

Figures (2)

  • Figure 1: Phases of our study. ORS connects experts with a trained crowd to perform real-world OSINT investigations described in Section \ref{['sec:osint-lab-course']}. The study sessions facilitate synchronous collaboration through 4 chronological phases: (1) The investigator presents the investigation topic and lists tasks that relate to the 5 OSINT macrotasks (presented in Table \ref{['tab:tasks']}). (2) The authors collaborate with the investigator to assign tasks to teams and provide links for submission. (3) The crowd strategizes the division of work within their teams. (4) The crowd conducts investigations for the rest of the session, submitting their findings through a tailored Google Form. Responses are aggregated in a spreadsheet, allowing real-time expert feedback and guidance. (5) The expert debriefs the crowd by sharing their insights and high-level feedback.
  • Figure 2: Snapshot of the spreadsheet containing crowd submissions and corresponding expert feedback for session 4. This session was led by an investigative journalist (E4). The goal of investigation was to identify discourse around anti-vaccine protests occurring throughout Europe as well as the groups involved. The investigation involved the discovery task and verification tasks like geolocation and source analysis.