A Multi-Stage Workflow for the Review of Marketing Content with Reasoning Large Language Models

TL;DR

This paper addresses automatic compliance checking of marketing content under dynamic regulatory requirements using a two-stage workflow with a retrieval module and a violation-detection module powered by reasoning LLMs. The approach uses a retrieval stage to identify a minimal set of relevant requirements and a violation-detection stage with LoRA adapters fine-tuned via SFT or GRPO, optionally generating reasoning tokens. Key contributions include a knowledge-free document-level validation method, a comparison of SFT and GRPO, an evaluation of reasoning-token generation, and an analysis of GRPO reward functions, demonstrated with Recall@k metrics and efficiency gains. The results show GRPO with reasoning tokens provides the strongest performance on larger models and that BLEU-based reward signals can outperform binary accuracy rewards, enabling scalable, adaptable compliance checks for marketing teams.

Abstract

Reasoning Large Language Models (LLMs) have shown promising results when tasked with solving complex problems. In this paper, we propose and evaluate a multi-stage workflow that leverages the capabilities of fine-tuned reasoning LLMs to assist in the review process of marketing content, making sure they comply with a given list of requirements. The contributions of this paper are the following: (i) we present a novel approach -- that does not rely on any external knowledge representation -- for the automatic identification of compliance issues in textual content; (ii) compare the effectiveness of different fine-tuning strategies like Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO) in training models to solve this problem; (iii) we evaluate the effectiveness of training small LLMs to generate reasoning tokens before providing their final response; (iv) we evaluate how the choice and combinations of different reward functions affects the performance of a model trained with GRPO.

Figures (4)

• Figure 1: Architecture of the proposed multi-stage workflow for content validation: (i) the content to validate is compared to the list of available requirements and a subset of relevant requirements that may apply to this type of content is selected for the validation stage; (ii) the content is checked against each selected requirement and a boolean value is produced to indicate whether the content passes or fails the validation check, together with a justification for each detected violation.
• Figure 2: Workflow for the generation of violated content drafts.
• Figure 3: Workflow for the generation of content drafts without violations.
• Figure 4: Number of relevant requirements identified for each content draft.