ConstitutionalExperts: Training a Mixture of Principle-based Prompts

TL;DR

This work tackles the challenge of costly, opaque prompt engineering by introducing ConstitutionalExperts, a method that learns a set of principled prompts and uses a mixture-of-experts to route inputs to cluster-specific Experts. The approach incrementally mutates individual principles within each Expert and leverages an iterative feedback loop with LLMs to guide improvements, while routing at inference to the most semantically relevant Expert via cluster centroids. Empirically, ConstitutionalExperts outperforms state-of-the-art discrete prompt optimizers by an average of $10.9\%$ F1 across six datasets, and the MoE extension yields additional gains of about $2.0\%$ F1 on average, illustrating broad applicability and interpretability. The work suggests significant practical impact for robust, modular prompting and outlines future directions including broader NLP tasks and alternative routing strategies.

Abstract

Large language models (LLMs) are highly capable at a variety of tasks given the right prompt, but writing one is still a difficult and tedious process. In this work, we introduce ConstitutionalExperts, a method for learning a prompt consisting of constitutional principles (i.e. rules), given a training dataset. Unlike prior methods that optimize the prompt as a single entity, our method incrementally improves the prompt by surgically editing individual principles. We also show that we can improve overall performance by learning unique prompts for different semantic regions of the training data and using a mixture-of-experts (MoE) architecture to route inputs at inference time. We compare our method to other state of the art prompt-optimization techniques across six benchmark datasets. We also investigate whether MoE improves these other techniques. Our results suggest that ConstitutionalExperts outperforms other prompt optimization techniques by 10.9% (F1) and that mixture-of-experts improves all techniques, suggesting its broad applicability.

Figures (2)

• Figure 1: Overview of the ConstitutionalExperts training loop to create principle-based prompts. Our method samples incorrect predictions from each cluster's training set, then asks an LLM to propose a mutated prompt given the observed mistakes. Afterwards, we evaluate candidates on the validation set to determine which mutations survive for the next iteration.
• Figure 2: Full Constitutional Experts approach. First, (A) we embed and cluster the training data. (B) Then, for each cluster, we learn a Constitutional Expert (shown in Figure \ref{['fig:training-loop']}). (C) At inference, we compute the similarity between the test example and each cluster's centroid and route the task to the nearest expert.