Towards AI as Colleagues: Multi-Agent System Improves Structured Professional Ideation

TL;DR

The paper tackles the limitation of single-agent AI in joint problem-solving by introducing MultiColleagues, a multi-agent system where diverse AI personas participate as co-ideators. It evaluates this design against a single-agent baseline in a within-subjects study with 20 participants, using think-aloud protocols, post-task surveys, and qualitative analyses. Results show that multi-agent collaboration enhances social presence, expands creative exploration, and improves idea quality and novelty, while also increasing user agency and engagement through explicit divergent/convergent thinking and facilitated coordination. The authors derive design implications for proactive, multi-voiced AI colleagues, seamless conversation orchestration, recognizable AI identities, and transparent trust mechanisms, highlighting ethical considerations and future work on scalability and adaptability. Overall, the work demonstrates a concrete path from tools to collegial AI partners in ideation, with practical implications for designing trustworthy, collaborative generative systems.

Abstract

Most AI systems today are designed to manage tasks and execute predefined steps. This makes them effective for process coordination but limited in their ability to engage in joint problem-solving with humans or contribute new ideas. We introduce MultiColleagues, a multi-agent conversational system that shows how AI agents can act as colleagues by conversing with each other, sharing new ideas, and actively involving users in collaborative ideation. In a within-subjects study with 20 participants, we compared MultiColleagues to a single-agent baseline. Results show that MultiColleagues fostered stronger perceptions of social presence, produced ideas rated significantly higher in quality and novelty, and encouraged deeper elaboration. These findings demonstrate the potential of AI agents to move beyond process partners toward colleagues that share intent, strengthen group dynamics, and collaborate with humans to advance ideas.

Figures (8)

• Figure 1: System Workflow for Persona-Guided Discussions. This diagram illustrates the end-to-end workflow of the persona-guided discussion system across five steps. The process begins with preparing the discussion by selecting AI colleagues (Step 1). The internal system first builds a persona ranking prompt, next selects the next persona, then builds a persona prompt and generates the corresponding reply (Step 2). The generated output is presented to the user, and the system awaits user action (Step 3). The user performs one of three actions: type a response, continue with the system-generated reply, or call a facilitator for support (Step 4). All logs are stored in the database (Step 5).
• Figure 2: Usage flow of the Multi-agent Conversational System for co-ideation. (1) Users first select a team of persona experts. (2) They then discuss the problem, with personas contributing from their own perspectives. The system supports two thinking modes: Explore mode expands ideas broadly, while Focus mode refines and synthesizes them. A summary recap highlights key points, and facilitation is triggered either by user request or automatically by the system when guidance is needed.
• Figure 3: User Study Workflow.
• Figure 4: This figure compares discussion patterns between MultiColleagues and Baseline. Baseline generated main topics (0.78 vs. 0.36 per minute) and sub-topics (3.09 vs. 1.25 per minute) at a faster rate, but branching ratios were comparable. MultiColleagues invested significantly more time per topic (2.93 vs. 1.39 minutes) and per sub-topic (0.91 vs. 0.36 minutes), supporting a slower, more deliberate style that enabled sustained idea elaboration.
• Figure 5: Design Implications for Human–Multi-Agent Collaboration: From AI as Tools to AI as Colleagues. This table summarizes six design implications that guide the transition from single-agent, tool-like AI to multi-agent colleagues in collaborative ideation. Each implication highlights a key design consideration (left column) and contrasts current AI as task-oriented tools (middle) with envisioned dynamics of human–multi-agent collaboration, where AI act as proactive, accountable, and socially embedded colleagues (right).