Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MAP: Multi-user Personalization with Collaborative LLM-powered Agents

Christine Lee, Jihye Choi, Bilge Mutlu

TL;DR

This work tackles the challenge of personalizing AI in multi-user environments with conflicting directives. It introduces a user-centered three-stage workflow (Reflection, Analysis, Feedback) grounded in conflict-resolution theory and implements it via MAP, a multi-agent LLM-based system with Planner, Rule Retriever, and Rule Manager. An empirical evaluation contrasts MAP with a monolithic LLM, showing MAP achieves superior retrieval and conflict-resolution performance, complemented by a user study (n=12) that highlights usability, transparency, and the importance of user verification. The findings demonstrate the practicality of multi-agent collaboration for scalable, user-centered multi-user personalization and point to future reliability enhancements and scalability directions.

Abstract

The widespread adoption of Large Language Models (LLMs) and LLM-powered agents in multi-user settings underscores the need for reliable, usable methods to accommodate diverse preferences and resolve conflicting directives. Drawing on conflict resolution theory, we introduce a user-centered workflow for multi-user personalization comprising three stages: Reflection, Analysis, and Feedback. We then present MAP -- a \textbf{M}ulti-\textbf{A}gent system for multi-user \textbf{P}ersonalization -- to operationalize this workflow. By delegating subtasks to specialized agents, MAP (1) retrieves and reflects on relevant user information, while enhancing reliability through agent-to-agent interactions, (2) provides detailed analysis for improved transparency and usability, and (3) integrates user feedback to iteratively refine results. Our user study findings (n=12) highlight MAP's effectiveness and usability for conflict resolution while emphasizing the importance of user involvement in resolution verification and failure management. This work highlights the potential of multi-agent systems to implement user-centered, multi-user personalization workflows and concludes by offering insights for personalization in multi-user contexts.

MAP: Multi-user Personalization with Collaborative LLM-powered Agents

TL;DR

This work tackles the challenge of personalizing AI in multi-user environments with conflicting directives. It introduces a user-centered three-stage workflow (Reflection, Analysis, Feedback) grounded in conflict-resolution theory and implements it via MAP, a multi-agent LLM-based system with Planner, Rule Retriever, and Rule Manager. An empirical evaluation contrasts MAP with a monolithic LLM, showing MAP achieves superior retrieval and conflict-resolution performance, complemented by a user study (n=12) that highlights usability, transparency, and the importance of user verification. The findings demonstrate the practicality of multi-agent collaboration for scalable, user-centered multi-user personalization and point to future reliability enhancements and scalability directions.

Abstract

The widespread adoption of Large Language Models (LLMs) and LLM-powered agents in multi-user settings underscores the need for reliable, usable methods to accommodate diverse preferences and resolve conflicting directives. Drawing on conflict resolution theory, we introduce a user-centered workflow for multi-user personalization comprising three stages: Reflection, Analysis, and Feedback. We then present MAP -- a \textbf{M}ulti-\textbf{A}gent system for multi-user \textbf{P}ersonalization -- to operationalize this workflow. By delegating subtasks to specialized agents, MAP (1) retrieves and reflects on relevant user information, while enhancing reliability through agent-to-agent interactions, (2) provides detailed analysis for improved transparency and usability, and (3) integrates user feedback to iteratively refine results. Our user study findings (n=12) highlight MAP's effectiveness and usability for conflict resolution while emphasizing the importance of user involvement in resolution verification and failure management. This work highlights the potential of multi-agent systems to implement user-centered, multi-user personalization workflows and concludes by offering insights for personalization in multi-user contexts.

Paper Structure

This paper contains 41 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Interaction Design of MAP
  3. Single-user vs Multi-user Personalization
  4. Everyday Scenarios
  5. Multi-user Personalization Workflow: Reflection, Analysis, and Feedback
  6. MAP: Multi-Agent LLMs for Multi-User Personalization
  7. Stage 0: Data Ingestion.
  8. Stage 1: Reflection.
  9. Stage 2: Analysis.
  10. Stage 3: Feedback.
  11. Limitations of a Monolithic LLM Approach for Multi-User Personalization
  12. User Study of MAP
  13. Study Design
  14. Participant Demographics
  15. Findings
  16. ...and 26 more sections

Figures (2)

  • Figure 1: Our multi-agent system to support the proposed multi-user personalization workflow --- The system orchestrates specialized agents through three stages of our user-centered workflow—Reflection, Analysis, and Feedback—to retrieve user data, reason about personalization tasks, resolve conflicts, and incorporate user feedback.
  • Figure 2: Monolithic LLM-based approach ("LLM") vs. our multi-agent approach ("MAP") across the three multi-user personalization scenarios--- The evaluation measures each system's success (in %) of 1) completely retrieving all 60 user preferences and schedules to generate personalized action plans ("Retrieval"), and 2) identifying and resolving all 12 present conflicts among the users ("Conflict").