The cost of coordination can exceed the benefit of collaboration in performing complex tasks

TL;DR

It is found that the extent of training received by an individual, the complexity of the task at hand, and the desired performance indicator are all critical factors that need to be accounted for when weighing up the benefits of collective decision making.

Abstract

Humans and other intelligent agents often rely on collective decision making based on an intuition that groups outperform individuals. However, at present, we lack a complete theoretical understanding of when groups perform better. Here we examine performance in collective decision-making in the context of a real-world citizen science task environment in which individuals with manipulated differences in task-relevant training collaborated. We find 1) dyads gradually improve in performance but do not experience a collective benefit compared to individuals in most situations; 2) the cost of coordination to efficiency and speed that results when switching to a dyadic context after training individually is consistently larger than the leverage of having a partner, even if they are expertly trained in that task; and 3) on the most complex tasks having an additional expert in the dyad who is adequately trained improves accuracy. These findings highlight that the extent of training received by an individual, the complexity of the task at hand, and the desired performance indicator are all critical factors that need to be accounted for when weighing up the benefits of collective decision-making.

Figures (17)

• Figure 1: Experimental design. (A) Examples of the experimental task and illustration of the difference between the set of images seen in the general versus the targeted training condition. Instances of images classified in the testing stage are also provided. (B) Sequential schema of events in the Experiment. In the training stage, $T_1$ every participant classified images individually. In the testing stage $T_2$, participants were additionally assigned to either an individual or dyad condition. Wildcam Gorongosa imagery is reprinted under a CC BY 4.0 license with permission.
• Figure 2: Individual learning trajectories of citizen scientists. Trajectories are defined as the change in the proportion of correct classifications averaged across each of the five Wildcam Gorongosa tasks relative to the number of classifications made. Each learning trajectory is for an individual volunteer citizen scientist who accessed the Wildcam Gorongosa site prior to the time of the experiment. Note the logarithmic scale on the x-axis, hence for a small number of classifications, the curves take exotic shapes.
• Figure 3: Task complexity mediates performance. Data is combined across all solo and dyad grouping conditions for the testing stage. The more 'complex' the task, the greater the reduction in average accuracy (A) and average efficiency (B). The difference in experienced difficulty between the task with the lowest and highest complexity is very large: the average accuracy score dropped by nearly 50%. Error bars indicate the 95% confidence intervals.
• Figure 4: Individual training effectiveness. Performance changes in terms of the average change in efficiency across each of the five Wildcam Gorongosa tasks during the training ($T_1$) and testing stage ($T_2$) for General Solo (A), individuals who received general training during $T_1$, and Targeted Solo (B), individuals who received selective training. The dashed vertical line falling in $T_2$ separates both stages, which each have 3 data points. Error bars indicate one standard error of the mean.
• Figure 5: Individual and collective performance. Performance differences between individuals and dyads for the last third of $T_2$ in terms of the distribution and average change in pace (A-B), alongside accuracy (C-D) and efficiency (E-F) for the species identification task, considered the most complex. General Solo individuals are compared to General Dyads and Mixed Dyads (left panels) and Targeted Solo individuals are compared to Targeted Dyads and Mixed Dyads (right panels). Error bars indicate one standard error of the mean. Whiskers are 1.5 times the interquartile range. The solid black line inside each box indicates the median and the red circles do the mean.