Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unveiling the Inter-Related Preferences of Crowdworkers: Implications for Personalized and Flexible Platform Design

Senjuti Dutta, Rhema Linder, Alex C. Williams, Anastasia Kuzminykh, Scott Ruoti

TL;DR

The paper addresses the rigidity of crowdworking platforms by investigating how interrelated preferences for device type and work stage shape worker practices. Using a survey of 150 MTurk workers, the authors integrate qualitative coding with PCA-based dimensionality reduction and k-means clustering to identify three distinct worker clusters with varying openness to non-workstation devices. PCA components and statistical tests delineate the key features that separate clusters, while a decision-tree model highlights the most discriminative attributes. The resulting design guidelines advocate desktop-optimized interfaces, hybrid device workflows, and mobile-first capabilities to support personalized, flexible crowdsourcing environments, ultimately improving efficiency and worker experience across diverse device ecosystems.

Abstract

Crowdsourcing platforms have traditionally been designed with a focus on workstation interfaces, restricting the flexibility that crowdworkers need. Recognizing this limitation and the need for more adaptable platforms, prior research has highlighted the diverse work processes of crowdworkers, influenced by factors such as device type and work stage. However, these variables have largely been studied in isolation. Our study is the first to explore the interconnected variabilities among these factors within the crowdwork community. Through a survey involving 150 Amazon Mechanical Turk crowdworkers, we uncovered three distinct groups characterized by their interrelated variabilities in key work aspects. The largest group exhibits a reliance on traditional devices, showing limited interest in integrating smartphones and tablets into their work routines. The second-largest group also primarily uses traditional devices but expresses a desire for supportive tools and scripts that enhance productivity across all devices, particularly smartphones and tablets. The smallest group actively uses and strongly prefers non-workstation devices, especially smartphones and tablets, for their crowdworking activities. We translate our findings into design insights for platform developers, discussing the implications for creating more personalized, flexible, and efficient crowdsourcing environments. Additionally, we highlight the unique work practices of these crowdworker clusters, offering a contrast to those of more traditional and established worker groups.

Unveiling the Inter-Related Preferences of Crowdworkers: Implications for Personalized and Flexible Platform Design

TL;DR

The paper addresses the rigidity of crowdworking platforms by investigating how interrelated preferences for device type and work stage shape worker practices. Using a survey of 150 MTurk workers, the authors integrate qualitative coding with PCA-based dimensionality reduction and k-means clustering to identify three distinct worker clusters with varying openness to non-workstation devices. PCA components and statistical tests delineate the key features that separate clusters, while a decision-tree model highlights the most discriminative attributes. The resulting design guidelines advocate desktop-optimized interfaces, hybrid device workflows, and mobile-first capabilities to support personalized, flexible crowdsourcing environments, ultimately improving efficiency and worker experience across diverse device ecosystems.

Abstract

Crowdsourcing platforms have traditionally been designed with a focus on workstation interfaces, restricting the flexibility that crowdworkers need. Recognizing this limitation and the need for more adaptable platforms, prior research has highlighted the diverse work processes of crowdworkers, influenced by factors such as device type and work stage. However, these variables have largely been studied in isolation. Our study is the first to explore the interconnected variabilities among these factors within the crowdwork community. Through a survey involving 150 Amazon Mechanical Turk crowdworkers, we uncovered three distinct groups characterized by their interrelated variabilities in key work aspects. The largest group exhibits a reliance on traditional devices, showing limited interest in integrating smartphones and tablets into their work routines. The second-largest group also primarily uses traditional devices but expresses a desire for supportive tools and scripts that enhance productivity across all devices, particularly smartphones and tablets. The smallest group actively uses and strongly prefers non-workstation devices, especially smartphones and tablets, for their crowdworking activities. We translate our findings into design insights for platform developers, discussing the implications for creating more personalized, flexible, and efficient crowdsourcing environments. Additionally, we highlight the unique work practices of these crowdworker clusters, offering a contrast to those of more traditional and established worker groups.
Paper Structure (28 sections, 5 figures, 4 tables)

This paper contains 28 sections, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Use of Workstation in Different Stages of Crowdwork
  4. Use of Non-workstation Devices in Different Stages of Crowdwork
  5. Relation to This Work
  6. Methodology
  7. Analysis Method
  8. Describing Clusters Overall Approach
  9. Principal Component (PCA) Based Analysis
  10. Statistical Differences
  11. Decision Tree
  12. Findings
  13. Description of PCA Components
  14. PCA Component 1 -- Preference of Non-workstation devices for both Completion and Management
  15. PCA Component 2 -- Selective Device Utilization for HIT Management and Completion
  16. ...and 13 more sections

Figures (5)

  • Figure 1: Elbow plot which shows the optimized number of clusters are 3
  • Figure 2: Siloutte score plot which shows the optimized number of clusters at 3
  • Figure 3: Visualization of crowdworker clusters with the PCA component
  • Figure 4: Decision Tree displaying all features that differentiate crowdworker clusters.
  • Figure 5: Visualization of crowdworker clusters based on their enthusiasm for non-workstation devices for task completion and management