Item Region-based Style Classification Network (IRSN): A Fashion Style Classifier Based on Domain Knowledge of Fashion Experts

TL;DR

This work tackles fashion style classification by integrating global image cues with item-level region features through an item region-based style classification network (IRSN). IRSN employs item region pooling (IRP) to extract item-specific features, analyzes them with dedicated item encoders, and fuses them with global features via gated feature fusion (GFF), all within a dual-backbone framework that combines a domain-specific extractor and a CLIP-based general encoder. Across six backbones on FashionStyle14 and ShowniqV3, IRSN achieves an average accuracy improvement around 7% and up to ~15% on individual cases, with visualization showing clearer separation between visually similar styles. The approach demonstrates the value of incorporating item-level detail and vision-language pretraining for robust fashion style recognition, with broad applicability to backbone choices and potential for improved downstream fashion analysis tasks.

Abstract

Fashion style classification is a challenging task because of the large visual variation within the same style and the existence of visually similar styles. Styles are expressed not only by the global appearance, but also by the attributes of individual items and their combinations. In this study, we propose an item region-based fashion style classification network (IRSN) to effectively classify fashion styles by analyzing item-specific features and their combinations in addition to global features. IRSN extracts features of each item region using item region pooling (IRP), analyzes them separately, and combines them using gated feature fusion (GFF). In addition, we improve the feature extractor by applying a dual-backbone architecture that combines a domain-specific feature extractor and a general feature extractor pre-trained with a large-scale image-text dataset. In experiments, applying IRSN to six widely-used backbones, including EfficientNet, ConvNeXt, and Swin Transformer, improved style classification accuracy by an average of 6.9% and a maximum of 14.5% on the FashionStyle14 dataset and by an average of 7.6% and a maximum of 15.1% on the ShowniqV3 dataset. Visualization analysis also supports that the IRSN models are better than the baseline models at capturing differences between similar style classes.

Figures (10)

• Figure 1: Examples of fashion styles. (a) classic (gentleman), (b) minimalist (c) vacation, (d) retro, (e) girly, (f) feminine, (g) ethnic, and (h) feminine (with different appearance from (f)).
• Figure 2: The structure of IRSN. $(H, W)$ and $(h, w)$ denote the size of the input image and the feature map, respectively.
• Figure 3: An example of item segmentation result. (a) input image, (b)-(e) the masks of head, top, bottom, and shoes regions element-wise multiplied by the input image, respectively.
• Figure 4: Item Region Pooling. The binary mask generated by the item region segmentator is resized to the size of the feature map extracted by the backbone for domain-specific feature extraction and then multiplied to pool only the item area for each region.
• Figure 5: Adaptive average pooling from $c \times 3 \times 3$ to $c \times 3 \times 1$