Actionable Advice from Reviews via Mixture of LoRA Experts: A Two-LLM Pipeline for Issue Extraction and Business Recommendations

TL;DR

This work tackles converting user reviews into concrete, actionable business recommendations by proposing a modular two-LLM pipeline: an Issue LLM for extracting salient problems and themes, followed by an Advice LLM that generates targeted operational fixes conditioned on the issue representation. The Advice LLM leverages a mixture of LoRA adapters, enabling domain specialization across industries without full fine-tuning, and uses a gating mechanism to mix experts at the token level. Synthetic training data from Yelp reviews across airlines and restaurants, generated via large LLMs, supervise adapters trained on distinct domain narratives; the model is evaluated with a multidimensional eight-criterion rubric, emphasizing actionability, specificity, feasibility, impact, novelty, non-redundancy, bias, and clarity. Across both domains, the proposed approach outperforms prompting-based and single-adapter baselines on actionability and specificity, while maintaining favorable efficiency-quality trade-offs, and demonstrates cross-domain transfer benefits, albeit with a remaining tension between highly actionable advice and practical feasibility.

Abstract

Customer reviews contain detailed, domain specific signals about service failures and user expectations, but converting this unstructured feedback into actionable business decisions remains difficult. We study review-to-action generation: producing concrete, implementable recommendations grounded in review text. We propose a modular two-LLM framework in which an Issue model extracts salient issues and assigns coarse themes, and an Advice model generates targeted operational fixes conditioned on the extracted issue representation. To enable specialization without expensive full fine-tuning, we adapt the Advice model using a mixture of LoRA experts strategy: multiple low-rank adapters are trained and a lightweight gating mechanism performs token-level expert mixing at inference, combining complementary expertise across issue types. We construct synthetic review-issue-advice triples from Yelp reviews (airlines and restaurants) to supervise training, and evaluate recommendations using an eight dimension operational rubric spanning actionability, specificity, feasibility, expected impact, novelty, non-redundancy, bias, and clarity. Across both domains, our approach consistently outperforms prompting-only and single-adapter baselines, yielding higher actionability and specificity while retaining favorable efficiency-quality trade-offs.

Figures (5)

• Figure 1: Overview of the proposed multi-stage LLM pipeline. User reviews are processed by an Issue LLM to extract specific issues and their corresponding themes, which are then passed to an Advice LLM that generates targeted, actionable fixes.
• Figure 2: Architecture of the proposed gated multi-industry transformer for review to fix generation. User reviews are encoded into input embeddings that propagate through a pretrained transformer stack. Each transformer block augments the shared parameters ($\theta$) with industry-specific offsets $\Delta\theta(i)$, producing specialised representations $E_{\Delta \theta_i}$. A learnable gating function G adaptively combines these industry-conditioned outputs into a single fused representation, which is then passed through the remaining transformer layers to generate context- and domain-aligned fixes.
• Figure 3: Composite scores (0–100) for each model in the Airline and Restaurant industries. Points denote the industry-specific composite (mean across eight dimensions), and horizontal connectors link the same model across industries to highlight cross-domain differences.
• Figure 4: Airline industry: per-dimension score changes relative to the Base Model for Airline LoRA, Restaurant LoRA, and the Proposed Model. Each cell reports the signed difference in points (model - base), showing large gains in novelty (up to +35) and specificity (+10 to +20), but a consistent drop in feasibility (-40) across all adapted models; the Proposed Model also improves actionability (+5) and reading clarity (+5).
• Figure 5: Restaurant industry: per-dimension score changes relative to the Base Model for Airline LoRA, Restaurant LoRA, and the Proposed Model. Each cell reports the signed difference in points (model - base), highlighting that the Proposed Model delivers the largest gains in specificity (+28.6) and reading clarity (+30.4), while feasibility decreases across all models (-3.6 to -19.4).