Masked Language Prompting for Generative Data Augmentation in Few-shot Fashion Style Recognition
Yuki Hirakawa, Ryotaro Shimizu
TL;DR
This paper tackles fashion style recognition under limited supervision, where style concepts are subjective and hard to annotate. It introduces Masked Language Prompting (MLP), a fine-tuning-free data augmentation framework that masks parts of reference captions and uses large language models to generate diverse yet semantically coherent completions, which are then converted into images with text-to-image models. Using FashionStyle14 and a CLIP-based linear classifier, MLP consistently outperforms class-name and caption-based prompts, and yields additional gains when combined with RandAugment or AugMix, with diversity and distribution metrics indicating better alignment with real data. The results demonstrate that MLP provides a scalable, style-faithful augmentation strategy that improves few-shot performance and mitigates semantic ambiguity in fashion style recognition.
Abstract
Constructing dataset for fashion style recognition is challenging due to the inherent subjectivity and ambiguity of style concepts. Recent advances in text-to-image models have facilitated generative data augmentation by synthesizing images from labeled data, yet existing methods based solely on class names or reference captions often fail to balance visual diversity and style consistency. In this work, we propose \textbf{Masked Language Prompting (MLP)}, a novel prompting strategy that masks selected words in a reference caption and leverages large language models to generate diverse yet semantically coherent completions. This approach preserves the structural semantics of the original caption while introducing attribute-level variations aligned with the intended style, enabling style-consistent and diverse image generation without fine-tuning. Experimental results on the FashionStyle14 dataset demonstrate that our MLP-based augmentation consistently outperforms class-name and caption-based baselines, validating its effectiveness for fashion style recognition under limited supervision.