Style Transfer as Bias Mitigation: Diffusion Models for Synthetic Mental Health Text for Arabic
Saad Mankarious, Aya Zirikly
TL;DR
The paper tackles data scarcity and demographic bias in Arabic mental health text by proposing a pretrained-free diffusion-based style-transfer approach that transforms male-authored posts into female-styled content to augment underrepresented female data. It introduces five dataset configurations (D1–D5) capturing varying linguistic aspects of female expression and trains separate diffusion models for each, conditioning generation on existing male posts. Across evaluations, the method achieves high semantic fidelity (BERTScore $F1$ around $0.93$–$0.95$) while inducing meaningful surface-level stylistic changes and demonstrates plausible gender transformations, all without relying on pretrained LLMs. This diffusion-based framework offers a flexible, ethics-aware route for mitigating gender bias in low-resource mental health NLP and can be extended to other biases or languages.
Abstract
Synthetic data offers a promising solution for mitigating data scarcity and demographic bias in mental health analysis, yet existing approaches largely rely on pretrained large language models (LLMs), which may suffer from limited output diversity and propagate biases inherited from their training data. In this work, we propose a pretraining-free diffusion-based approach for synthetic text generation that frames bias mitigation as a style transfer problem. Using the CARMA Arabic mental health corpus, which exhibits a substantial gender imbalance, we focus on male-to-female style transfer to augment underrepresented female-authored content. We construct five datasets capturing varying linguistic and semantic aspects of gender expression in Arabic and train separate diffusion models for each setting. Quantitative evaluations demonstrate consistently high semantic fidelity between source and generated text, alongside meaningful surface-level stylistic divergence, while qualitative analysis confirms linguistically plausible gender transformations. Our results show that diffusion-based style transfer can generate high-entropy, semantically faithful synthetic data without reliance on pretrained LLMs, providing an effective and flexible framework for mitigating gender bias in sensitive, low-resource mental health domains.