Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Spark Effect: On Engineering Creative Diversity in Multi-Agent AI Systems

Alexander Doudkin, Anton Voelker, Friedrich von Borries

TL;DR

The paper addresses the challenge of producing diverse, high-quality creative concepts in production LLM workflows, where single-agent prompts yield homogenous outputs. It proposes Spark, a library of persona-conditioned LLM agents, augmented by a retrieval-augmented grounding pipeline and a structured evaluation protocol calibrated against human judgments. Empirical results show substantial diversity gains over a baseline and substantial convergence toward human-level diversity, with caveats around evaluator bias and persona drift. The work demonstrates practical operational benefits, outlines limitations, and commits to open-sourcing benchmarks and persona libraries to advance robustness in creative AI practices.

Abstract

Creative services teams increasingly rely on large language models (LLMs) to accelerate ideation, yet production systems often converge on homogeneous outputs that fail to meet brand or artistic expectations. Art of X developed persona-conditioned LLM agents -- internally branded as "Sparks" and instantiated through a library of role-inspired system prompts -- to intentionally diversify agent behaviour within a multi-agent workflow. This white paper documents the problem framing, experimental design, and quantitative evidence behind the Spark agent programme. Using an LLM-as-a-judge protocol calibrated against human gold standards, we observe a mean diversity gain of +4.1 points (on a 1-10 scale) when persona-conditioned Spark agents replace a uniform system prompt, narrowing the gap to human experts to 1.0 point. We also surface evaluator bias and procedural considerations for future deployments.

The Spark Effect: On Engineering Creative Diversity in Multi-Agent AI Systems

TL;DR

The paper addresses the challenge of producing diverse, high-quality creative concepts in production LLM workflows, where single-agent prompts yield homogenous outputs. It proposes Spark, a library of persona-conditioned LLM agents, augmented by a retrieval-augmented grounding pipeline and a structured evaluation protocol calibrated against human judgments. Empirical results show substantial diversity gains over a baseline and substantial convergence toward human-level diversity, with caveats around evaluator bias and persona drift. The work demonstrates practical operational benefits, outlines limitations, and commits to open-sourcing benchmarks and persona libraries to advance robustness in creative AI practices.

Abstract

Creative services teams increasingly rely on large language models (LLMs) to accelerate ideation, yet production systems often converge on homogeneous outputs that fail to meet brand or artistic expectations. Art of X developed persona-conditioned LLM agents -- internally branded as "Sparks" and instantiated through a library of role-inspired system prompts -- to intentionally diversify agent behaviour within a multi-agent workflow. This white paper documents the problem framing, experimental design, and quantitative evidence behind the Spark agent programme. Using an LLM-as-a-judge protocol calibrated against human gold standards, we observe a mean diversity gain of +4.1 points (on a 1-10 scale) when persona-conditioned Spark agents replace a uniform system prompt, narrowing the gap to human experts to 1.0 point. We also surface evaluator bias and procedural considerations for future deployments.
Paper Structure (23 sections, 1 figure, 2 tables)

This paper contains 23 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Setting
  4. Creative Task Portfolio
  5. Baseline System
  6. Persona-Conditioned LLM Agent Library
  7. Spark Agent Automation Pipeline
  8. Persona Library Employed in Benchmarking
  9. Evaluation Methodology
  10. LLM-as-a-Judge Protocol
  11. Calibration.
  12. Instrumentation.
  13. Spark Agents vs. Baseline Experiments
  14. Results
  15. Aggregate Diversity Gains
  16. ...and 8 more sections

Figures (1)

  • Figure 1: Spark effect snapshot. Spark agents deliver a $+4.1$ point improvement over the pre-Spark specialised agent average, with scores clipped to the evaluator's 1--10 rubric.