Personality Pairing Improves Human-AI Collaboration

TL;DR

It is demonstrated that human-AI personality alignment significantly improves collaboration, productivity, and performance and lay a foundation for future research on improving human-AI collaboration through AI personalization.

Abstract

Here we ask how AI agent "personalities" interact with human personalities, and other traits, to shape human-AI collaboration, productivity and performance. To estimate these relationships, we conducted a large-scale preregistered randomized experiment that paired 1,258 participants with AI agents that were prompted to exhibit varying levels of the Big Five personality traits. These human-AI teams produced 7,266 display ads for a real think tank, and the quality of these ads was evaluated by 1,995 independent human raters as well as in a field experiment conducted on X, which generated nearly 5 million impressions. We found, first, that personality pairing impacted teamwork quality. For example, neurotic AI improved teamwork for agreeable humans but impaired it for conscientious humans. Second, we found productivity effects of personality pairing and a "productivity-performance trade-off" in which certain pairings (e.g., agreeable human with neurotic AI) produced fewer ads but of higher quality. Third, personality pairing influenced ad quality and performance. For example, quality improved when open humans were paired with conscientious AI and when conscientious humans were paired with disagreeable AI. Some of these pairing effects were "jagged" in that they varied across text and visual tasks. For example open humans produced higher quality images but lower quality text when paired with agreeable AI. Pairing effects were also present in other human traits, like country of origin. For example, extroverted AI improved quality for Latin American workers, but degraded quality for East Asian workers. These findings demonstrate that human-AI personality alignment significantly improves collaboration, productivity, and performance and lay a foundation for future research on improving human-AI collaboration through AI personalization.

Figures (7)

• Figure 1: The experimental design for investigating human-AI collaboration. (A) Participants are randomized into collaborating with either a human partner or an AI agent. (B) AI agents are randomly assigned prompts to induce low or high levels of the Big Five personality traits. (C) Pairs collaborate in a real-time collaborative workspace to create display ads.
• Figure 2: Regression coefficients for human-AI personality interaction terms for teamwork quality ($^{*}$p$<$0.05) hoegl2001. Error bars show 95% confidence intervals. See SI Section H for the full regression tables.
• Figure 3: Regression coefficients for human-AI personality interaction terms on ad text quality, image quality, and click likelihood ($^{*}$p$<$0.05). Error bars show 95% confidence intervals. See SI Section H for the full regression table.
• Figure 4: Regression coefficients for interaction terms of a participants' country of birth and AI personality on ad quality measures ($^{*}$p$<$0.05). Error bars show 95% confidence intervals. See SI Section H for the full regression table.
• Figure 5: Regression coefficients for human-AI personality interaction terms and ad quality ratings on click-through rates (CTR; basis points) and cost-per-click (CPC; $) and view-through rates (VTR; %) ($^{*}$p$<$0.05). Higher values indicate better performance for CTR and VTR, whereas lower values indicate better performance for CPC. Error bars show 95% confidence intervals. See SI Section K for the full regression tables.