Understanding Human-Multi-Agent Team Formation for Creative Work

TL;DR

Understanding Human-Multi-Agent Team Formation for Creative Work investigates how humans form HMATs to collaborate with multiple AI agents in creative workflows. The authors introduce CrafTeam, a technology probe, to systematically configure five HMAT formation dimensions and to study their impact on ideation through iterative cycles. In a study with 12 design practitioners, the findings show that while participants initially let agents autonomously generate ideas, they ultimately orchestrate the agents themselves to avoid unproductive loops, highlighting the need for human-centered HMAT formation. The results yield design considerations and a research agenda for scalable, human-aligned HMATs, including multi-party communication and progressive team evolution.

Abstract

Team-based collaboration is a cornerstone of modern creative work. Recent advances in generative AI open possibilities for humans to collaborate with multiple AI agents in distinct roles to address complex creative workflows. Yet, how to form Human-Multi-Agent Teams (HMATs) is underexplored, especially given that inter-agent interactions increase complexity and the risk of unexpected behaviors. In this exploratory study, we aim to understand how to form HMATs for creative work using CrafTeam, a technology probe that allows users to form and collaborate with their teams. We conducted a study with 12 design practitioners, in which participants iterated through a three-step cycle: forming HMATs, ideating with their teams, and reflecting on their teams' ideation. Our findings reveal that while participants initially attempted autonomous team operations, they ultimately adopted team formations in which they directly orchestrated agents. We discuss design considerations for HMAT formation that humans can effectively orchestrate multiple agents.

Figures (8)

• Figure 1: Overall flow of CrafTeam. CrafTeam supports three repeating phases—forming HMATs, ideating within HMATs, and reflecting on the team's ideation.
• Figure 2: Simplified Team Formation Interface of CrafTeam. (1) The user sets the team name and ideation topic and chooses the team size. (2) The user configures a networked team structure by linking members as hierarchical or peer. (3) The user assigns roles to each member—Idea Generation, Idea Evaluation, Feedback, and Request. (4) The user creates member personas in a resume-like form aligned with roles. (5) The user establishes the team's shared mental models.
• Figure 3: Ideation Interface of CrafTeam. (A) Idea Tab displays all team-generated ideas. It allows users to generate new ideas through the "Create New Idea" button and provides detailed views via the "See more details" option, which reveals full idea content and enables updates or evaluations (shown as Fig \ref{['fig:crafteam_idea_representation_UI']}). (B) Team Status Tab displays the team structure and member information, including the roles assigned to each member, the relationships between team members, and the current status of the AI agents. (C) Team Chat Window displays real-time interaction logs, including feedback sessions between team members. (D) Chat Input Field enables users to provide feedback or make requests to AI agents.
• Figure 4: Idea Card in CrafTeam. (A) The main view displays the idea representation, including Object, Function, Behavior, and Structure. (B) The Evaluation List displays team members' ratings and comments on the idea. (C) The Evaluate Idea Tab allows users to provide their own ratings and comments on the idea.
• Figure 5: Reflection interface of CrafTeam. The interface consists of five main panels: (i) The Session Summary Panel displays team information and session metrics. (ii) The Members Panel shows each member's persona along with their per-role action counts. (iii) The Team Relationship Panel renders the team network, and when Feedback or Request mode is toggled, it visualizes the flow of these interactions between members. (iv) The Action Timeline Panel presents a chronologically ordered list of actions, featuring an attendee filter and expandable entries for detailed information. (v) The Generated Ideas Panel lists all ideas with their scores and metadata, where each item can be expanded to reveal detailed information.