A Multi-Agent System for Generating Actionable Business Advice

TL;DR

The paper tackles turning large-scale, free-form customer reviews into actionable managerial guidance by proposing a four-component multi-agent framework that distills corpora into representative issues, generates and iteratively refines interventions, and ranks them by practicality and feasibility. Across three service domains and multiple model families, the approach yields consistently high actionability, specificity, and non-redundancy, with medium-sized models approaching the performance of large-scale frameworks. Ablation studies demonstrate the value of clustering, issue extraction, evaluation, and ranking components for stability and structured decision-making. The work advances review mining from descriptive insights to prescriptive strategy, while acknowledging limitations such as potential evaluation biases and the need for broader domain validation and human-in-the-loop assessment.

Abstract

Customer reviews contain rich signals about product weaknesses and unmet user needs, yet existing analytic methods rarely move beyond descriptive tasks such as sentiment analysis or aspect extraction. While large language models (LLMs) can generate free-form suggestions, their outputs often lack accuracy and depth of reasoning. In this paper, we present a multi-agent, LLM-based framework for prescriptive decision support, which transforms large scale review corpora into actionable business advice. The framework integrates four components: clustering to select representative reviews, generation of advices, iterative evaluation, and feasibility based ranking. This design couples corpus distillation with feedback driven advice refinement to produce outputs that are specific, actionable, and practical. Experiments across three service domains and multiple model families show that our framework consistently outperform single model baselines on actionability, specificity, and non-redundancy, with medium sized models approaching the performance of large model frameworks.

Figures (6)

• Figure 1: An example of the recommendation-evaluation loop. Given an issue and an initial advice proposal, the Evaluation Agent assigns rubric scores and textual feedback, which the Recommendation Agent uses to generate improved advice in subsequent iteration.
• Figure 2: Overview of the proposed multi-agent workflow: The Clustering agent embeds and clusters reviews to select representative reviews, the Issue agent groups them into higher-level issues, Recommendation–Evaluation agents iteratively refine candidate interventions, and the Ranking agent selects the best advice.
• Figure 3: An example of the review-to-advice pipeline. A representative review may contain multiple issues (e.g., long wait times, lack of customer service and excessive charges). The Issue Agent extracts and maps all detected issues to one or more themes; for clarity, only one issue is shown in this figure for demonstration. For each extracted issue, multiple candidate advices are generated and iteratively refined, and the Ranking Agent selects the best refined advice as the final Advice(per issue).
• Figure 4: Performance of the three multi-agent system across the three industries. Top row: average composite recommendation quality scores (0–100) for automotive, restaurant, and hospitality. Bottom row: heatmaps showing per-dimension scores for each ensemble in each domain.
• Figure 5: Performance of the vanilla single model baselines across the three industries. Top row: average composite recommendation-quality scores (0–100) for Llama-3.3-70B, gpt-oss-120B, and DeepSeek-r1-70B on automotive, restaurant, and hospitality datasets. Bottom row: corresponding heatmaps showing per dimension scores for each model in each domain.