Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs

Parsa Hejabi, Elnaz Rahmati, Alireza S. Ziabari, Morteza Dehghani

TL;DR

Flip-Flop Consistency ($F^2C$) addresses semantic inconsistency in LLMs caused by prompt perturbations using an unsupervised training framework that combines Consensus Cross-Entropy with a targeted representation alignment loss. By deriving a majority-vote pseudo-label across prompt variations and selectively aligning non-majority predictions toward this consensus, $F^2C$ improves both agreement across formats and task performance. Across 11 datasets and multiple generalization scenarios, $F^2C$ consistently increases observed agreement and mean F1 while reducing variance, outperforming swarm distillation and CCE-alone baselines, and showing favorable out-of-domain and unseen-variation generalization. The approach requires no gold labels and demonstrates practical impact for robust, reliable LLM deployment under varied prompting, with code available for replication and extension to generation settings.

Abstract

Large Language Models (LLMs) often produce inconsistent answers when faced with different phrasings of the same prompt. In this paper, we propose Flip-Flop Consistency ($F^2C$), an unsupervised training method that improves robustness to such perturbations. $F^2C$ is composed of two key components. The first, Consensus Cross-Entropy (CCE), uses a majority vote across prompt variations to create a hard pseudo-label. The second is a representation alignment loss that pulls lower-confidence and non-majority predictors toward the consensus established by high-confidence, majority-voting variations. We evaluate our method on 11 datasets spanning four NLP tasks, with 4-15 prompt variations per dataset. On average, $F^2C$ raises observed agreement by 11.62%, improves mean $F_1$ by 8.94%, and reduces performance variance across formats by 3.29%. In out-of-domain evaluations, $F^2C$ generalizes effectively, increasing $\overline{F_1}$ and agreement while decreasing variance across most source-target pairs. Finally, when trained on only a subset of prompt perturbations and evaluated on held-out formats, $F^2C$ consistently improves both performance and agreement while reducing variance. These findings highlight $F^2C$ as an effective unsupervised method for enhancing LLM consistency, performance, and generalization under prompt perturbations. Code is available at https://github.com/ParsaHejabi/Flip-Flop-Consistency-Unsupervised-Training-for-Robustness-to-Prompt-Perturbations-in-LLMs.

Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs

TL;DR

Flip-Flop Consistency () addresses semantic inconsistency in LLMs caused by prompt perturbations using an unsupervised training framework that combines Consensus Cross-Entropy with a targeted representation alignment loss. By deriving a majority-vote pseudo-label across prompt variations and selectively aligning non-majority predictions toward this consensus, improves both agreement across formats and task performance. Across 11 datasets and multiple generalization scenarios, consistently increases observed agreement and mean F1 while reducing variance, outperforming swarm distillation and CCE-alone baselines, and showing favorable out-of-domain and unseen-variation generalization. The approach requires no gold labels and demonstrates practical impact for robust, reliable LLM deployment under varied prompting, with code available for replication and extension to generation settings.

Abstract

Large Language Models (LLMs) often produce inconsistent answers when faced with different phrasings of the same prompt. In this paper, we propose Flip-Flop Consistency (), an unsupervised training method that improves robustness to such perturbations. is composed of two key components. The first, Consensus Cross-Entropy (CCE), uses a majority vote across prompt variations to create a hard pseudo-label. The second is a representation alignment loss that pulls lower-confidence and non-majority predictors toward the consensus established by high-confidence, majority-voting variations. We evaluate our method on 11 datasets spanning four NLP tasks, with 4-15 prompt variations per dataset. On average, raises observed agreement by 11.62%, improves mean by 8.94%, and reduces performance variance across formats by 3.29%. In out-of-domain evaluations, generalizes effectively, increasing and agreement while decreasing variance across most source-target pairs. Finally, when trained on only a subset of prompt perturbations and evaluated on held-out formats, consistently improves both performance and agreement while reducing variance. These findings highlight as an effective unsupervised method for enhancing LLM consistency, performance, and generalization under prompt perturbations. Code is available at https://github.com/ParsaHejabi/Flip-Flop-Consistency-Unsupervised-Training-for-Robustness-to-Prompt-Perturbations-in-LLMs.
Paper Structure (29 sections, 1 theorem, 16 equations, 6 figures, 3 tables, 1 algorithm)

This paper contains 29 sections, 1 theorem, 16 equations, 6 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Evaluating Consistency in LLMs
  4. Improving Consistency in LLMs
  5. Method
  6. Problem Formulation
  7. Per-variation scoring.
  8. Consensus Cross-Entropy
  9. Consensus label.
  10. Loss.
  11. Flip-Flop Consistency
  12. Total loss and hyperparameters.
  13. Metrics
  14. Experimental Setup
  15. Datasets
  16. ...and 14 more sections

Key Result

Theorem A.1

Let $Y$ be finite. For a fixed input with $K$ semantically equivalent formats, let $q_1,\dots,q_K \in \Delta^{|Y|}$ be teacher distributions (treated as constants) and let $p_j(\theta)\in \Delta^{|Y|}$ be the student for format $j$. For weights $w_i^{(j)}\ge 0$ with $\sum_{i=1}^K w_i^{(j)}=1$, defin Then, for each $j$, and the last sum is constant in $\theta$. Hence $\nabla_\theta$ of the left si

Figures (6)

  • Figure 1: Our method aligns representations of input variations to promote consistency. To this end, we minimize the JS divergence among datapoints within the high-confidence consensus group, and the KL divergence between all other datapoints and that group.
  • Figure 2: Top: $\overline{F_1}$ with shaded $\sigma_{F_1}$ on the held-out prompt formats. Bottom: observed agreement $P_o$ on the same held-out sets. The x-axis is the number of training formats ($K$); "Base Model" is the untrained baseline.
  • Figure 3: Per-dataset distribution of $F_1$ across prompt variations for the Qwen2.5-3B-Instruct.
  • Figure 4: Cross-dataset transfer for observed agreement $P_o$. Each cell shows $\Delta P_o$ relative to the base model (train on rows, evaluate on columns). Green indicates improvement; red indicates degradation.
  • Figure 5: $\Delta\overline{F_1}$: Cross-dataset transfer under F$^2$C. Each cell shows the change relative to the base model when training on the row dataset and evaluating on the column dataset. Green indicates improvement; red indicates degradation.
  • ...and 1 more figures

Theorems & Definitions (2)

  • Theorem A.1: Mixture-teacher decomposition
  • proof