Evaluating Collaborative and Autonomous Agents in Data-Stream-Supported Coordination of Mobile Crowdsourcing

TL;DR

The paper tackles high failure rates in data-stream-supported mobile crowdsourcing by combining continuous delivery-outcome prediction with two coordination paradigms: collaborative transfers among non-autonomous workers and market-based transfers among autonomous workers. It develops a data-stream learning framework to predict delivery success, and demonstrates substantial improvements in service quality through agent-based crowdshipping simulations and ATT auctions in crowdsensing. Key contributions include a general prediction model with $DELAYED$ vs $NOT_DELAYED$, an efficient collaborative transfer algorithm leveraging $prob_{delay}$, and a market-based auction mechanism with expected utility calculations for autonomous agents. The results indicate that outcome prediction coupled with peer-to-peer transfers can significantly reduce delays and improve profits, offering practical guidance for designing robust, autonomy-respecting mobile crowdsourcing systems.

Abstract

Mobile crowdsourcing refers to systems where the completion of tasks necessarily requires physical movement of crowdworkers in an on-demand workforce. Evidence suggests that in such systems, tasks often get assigned to crowdworkers who struggle to complete those tasks successfully, resulting in high failure rates and low service quality. A promising solution to ensure higher quality of service is to continuously adapt the assignment and respond to failure-causing events by transferring tasks to better-suited workers who use different routes or vehicles. However, implementing task transfers in mobile crowdsourcing is difficult because workers are autonomous and may reject transfer requests. Moreover, task outcomes are uncertain and need to be predicted. In this paper, we propose different mechanisms to achieve outcome prediction and task coordination in mobile crowdsourcing. First, we analyze different data stream learning approaches for the prediction of task outcomes. Second, based on the suggested prediction model, we propose and evaluate two different approaches for task coordination with different degrees of autonomy: an opportunistic approach for crowdshipping with collaborative, but non-autonomous workers, and a market-based model with autonomous workers for crowdsensing.

Figures (6)

• Figure S1: F1-score convergence for random forest (RF), Hoeffding (HT), and KNN.
• Figure S2: Agent-based simulator at work.
• Figure S3: Task neighborhood example.
• Figure S4: Delays under four strategies in three crowdsensing scenarios.
• Figure S5: The observed profits under four strategies in three crowdsensing scenarios.