FAME: Fairness-aware Attention-modulated Video Editing

TL;DR

FAME tackles fairness in training-free video editing by injecting fairness cues without retraining, using soft debiasing prompt encoding and attention modulation in both temporal self-attention and cross-attention. It introduces FairVE, a benchmarking dataset for fairness in frame-level VE with eight professions across genders. The approach preserves prompt semantics and temporal coherence while reducing gender bias, validated by automatic CLIP-based metrics and human studies, outperforming baselines such as FateZero, TokenFlow, and VideoGrain. The work demonstrates that fairness can be controlled at inference-time with minimal semantic disruption, enabling practical deployment of fair VE systems.

Abstract

Training-free video editing (VE) models tend to fall back on gender stereotypes when rendering profession-related prompts. We propose \textbf{FAME} for \textit{Fairness-aware Attention-modulated Video Editing} that mitigates profession-related gender biases while preserving prompt alignment and temporal consistency for coherent VE. We derive fairness embeddings from existing minority representations by softly injecting debiasing tokens into the text encoder. Simultaneously, FAME integrates fairness modulation into both temporal self attention and prompt-to-region cross attention to mitigate the motion corruption and temporal inconsistency caused by directly introducing fairness cues. For temporal self attention, FAME introduces a region constrained attention mask combined with time decay weighting, which enhances intra-region coherence while suppressing irrelevant inter-region interactions. For cross attention, it reweights tokens to region matching scores by incorporating fairness sensitive similarity masks derived from debiasing prompt embeddings. Together, these modulations keep fairness-sensitive semantics tied to the right visual regions and prevent temporal drift across frames. Extensive experiments on new VE fairness-oriented benchmark \textit{FairVE} demonstrate that FAME achieves stronger fairness alignment and semantic fidelity, surpassing existing VE baselines.

Figures (5)

• Figure 1: Training-free VE methods (e.g., TokenFlow geyer2023tokenflow) fail to correct gender stereotypes in profession related prompts, as shown in the 1st row, even with debiasing prompt, causing motion corruption, with missing actions, ghosting, and temporal discontinuity. Videograin yang_videograin_2025 partly reduces bias but still retains stereotypical features such as outfit, illustrated in the 2nd row. In contrast, FAME combines soft prompt encoding and attention modulation to remove both profession bias and motion corruption, generating coherent videos with accurate debiased profession details.
• Figure 2: Framework of FAME. In the denoising process, the soft debiasing prompt encoding module will encode the debiasing prompt ${\boldsymbol{\mathbf{P}}}_{\texttt{tar}}$ explicitly and fuse the self-attention map ${\boldsymbol{\mathbf{Q}}}{\boldsymbol{\mathbf{K}}}^{T}$ with time decay ${\boldsymbol{\mathbf{S}}}$ and element weight ${\boldsymbol{\mathbf{M}}}$ with cross-attention map reweighting.
• Figure 3: Cross-Attention Modulation Effect. Visualization of attention maps corresponding to the token $\texttt{CEO}$ under different ablation settings. The baseline model struggles to separate foreground(human) and background(tree), often overlooking regions related to fairness. Cross-attention reweighting helps adjust areas like outfit to better preserve identity features.
• Figure 4: Comparison of different prompt injection methods. Direct addition leads to semantic confusion, while our masked fusion retains structure and enhances fairness controllability.
• Figure 5: Prompt Responsiveness Test. We evaluate three training-free VE models, including FateZero, TokenFlow, and VideoGrain, on a single example ${\boldsymbol{\mathbf{P}}}_{\texttt{ref}}=\texttt{"A man is playing tennis"}$ to assess their responsiveness to debiasing prompt. The top row shows the input reference frames. Rows 2–4 display results using different prompts on several baselines, including FateZero qi_fatezero_2023, TokenFlow geyer2023tokenflow, and VideoGrain yang_videograin_2025, while the 5th row shows the results with the original editing prompt ${\boldsymbol{\mathbf{P}}}_{\texttt{tar}}=\texttt{"A teacher is playing tennis"}$ by our FAME. In rows 2–4, the left column shows the editing performance with the original prompt, and the right column shows the editing performance guided by the directly injected prompt ${\boldsymbol{\mathbf{P}}}_{\texttt{tar}}=\texttt{"A male teacher is playing tennis"}$ as the debiasing prompt. In the last row, our FAME can generate the female version (left column) and the debiased male content (right column). In the 2nd and 3rd rows, FateZero and TokenFlow exhibit a mismatch between the debiasing prompt and the visual semantics, indicating degraded video fidelity. In the 4th row, VideoGrain shows that directly injecting debiasing prompts often leads to semantic degradation in generated frames. This suggests that fairness-sensitive prompts may disrupt the alignment between the prompt and visual output. In the last row, our FAME can generate different gender-profession content even with the original prompt.