Multi-team Formation System for Collaborative Crowdsourcing
Ryota Yamamoto, Kazushi Okamoto
TL;DR
The paper tackles multi-task crowdsourcing by formulating a density-based team-compatibility objective over a social network and proposing a SAT-enabled simulated-annealing algorithm to form concurrent teams under skill, budget, and size constraints. It combines a smoothing mechanism to cope with sparsity and uses Gaussian process regression with Bayesian optimization to analyze how eight experimental parameters and hyperparameters affect performance. Empirical results show the proposed method outperforms a hill-climbing baseline, with the strongest gains near a cooling factor of $\alpha \approx 0.9$ and little benefit from smoothing ($\beta$). The work advances multi-team formation in collaborative crowdsourcing and highlights directions for fairness with new workers and more efficient feasibility solvers.
Abstract
For complex crowdsourcing tasks that require collaboration between multiple individuals, teams should be formed by considering both worker compatibility and expertise. Furthermore, the nature of crowdsourcing dictates the budget for tasks and workers' remuneration, and excessively large team sizes may reduce collaborative performance. To address these challenges, we propose a heuristic optimization algorithm that leverages social network information to simultaneously form teams with optimized worker compatibility for multiple tasks. In our approach, historical collaboration is represented as a social network, where the edge weights correspond to explicit ratings of worker compatibility. In a simulation experiment using synthetic data, we applied Gaussian process regression to examine the relationship between eight experimental parameters and evaluation values, thereby analyzing the output of the proposed algorithm. To generate the necessary data for regression, we ran the proposed algorithm with experimental parameters that were sequentially estimated using Bayesian optimization. Our experiments revealed that the evaluation values were extremely low when the team size limit, the degree mean of the social network, and the task budget were set to low values. The results also indicate that the proposed algorithm outperformed the hill-climbing method under almost all experimental conditions. In addition, the highest evaluation values were achieved when the simulated annealing temperature decrease rate was approximately 0.9, while smoothing the objective function proved ineffective.