Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The impact of abstract and object tags on image privacy classification

Darya Baranouskaya, Andrea Cavallaro

TL;DR

The paper investigates whether abstract or concrete tag types are more informative for image privacy classification. Using ClarifAI-generated tags and a controlled feature-selection pipeline, it compares abstract, concrete, and combined tag representations across three privacy datasets, varying the number of tags per image. Key findings show abstract tags excel when tag budgets are small and subjectivity is high, while larger tag budgets reduce the advantage of abstract information, allowing concrete or mixed tags to perform comparably. The analysis also reveals limited direct co-occurrence between abstract and concrete tags, suggesting that combined tag sets capture complementary cues when abundant tagging is available. Practically, the work guides the design of interpretable privacy classifiers by incorporating abstract concepts, especially for subjective tasks, while demonstrating budget-driven trade-offs between tag types.

Abstract

Object tags denote concrete entities and are central to many computer vision tasks, whereas abstract tags capture higher-level information, which is relevant for tasks that require a contextual, potentially subjective scene understanding. Object and abstract tags extracted from images also facilitate interpretability. In this paper, we explore which type of tags is more suitable for the context-dependent and inherently subjective task of image privacy. While object tags are generally used for privacy classification, we show that abstract tags are more effective when the tag budget is limited. Conversely, when a larger number of tags per image is available, object-related information is as useful. We believe that these findings will guide future research in developing more accurate image privacy classifiers, informed by the role of tag types and quantity.

The impact of abstract and object tags on image privacy classification

TL;DR

The paper investigates whether abstract or concrete tag types are more informative for image privacy classification. Using ClarifAI-generated tags and a controlled feature-selection pipeline, it compares abstract, concrete, and combined tag representations across three privacy datasets, varying the number of tags per image. Key findings show abstract tags excel when tag budgets are small and subjectivity is high, while larger tag budgets reduce the advantage of abstract information, allowing concrete or mixed tags to perform comparably. The analysis also reveals limited direct co-occurrence between abstract and concrete tags, suggesting that combined tag sets capture complementary cues when abundant tagging is available. Practically, the work guides the design of interpretable privacy classifiers by incorporating abstract concepts, especially for subjective tasks, while demonstrating budget-driven trade-offs between tag types.

Abstract

Object tags denote concrete entities and are central to many computer vision tasks, whereas abstract tags capture higher-level information, which is relevant for tasks that require a contextual, potentially subjective scene understanding. Object and abstract tags extracted from images also facilitate interpretability. In this paper, we explore which type of tags is more suitable for the context-dependent and inherently subjective task of image privacy. While object tags are generally used for privacy classification, we show that abstract tags are more effective when the tag budget is limited. Conversely, when a larger number of tags per image is available, object-related information is as useful. We believe that these findings will guide future research in developing more accurate image privacy classifiers, informed by the role of tag types and quantity.

Paper Structure

This paper contains 8 sections, 5 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Data
  3. Tag concreteness
  4. Tag representation for classifier
  5. Experiments
  6. Model trained on different types of tags
  7. Analysis of tags co-occurrence
  8. Conclusion

Figures (2)

  • Figure 1: Tag concreteness and their $\chi^2$ score with image privacy labels in the PrivacyAlert (a) and VISPR (b) datasets. The transparency of the dots reflects the tag frequency with which the tag appears in the dataset (the more transparent, the less frequent). We visualised the tag names of a few sample points. Both abstract and concrete tag sets have tags with high $\chi^2$ scores, signalling that tags of different abstractness are important for image privacy.
  • Figure 2: The achieved F1-macro (mean and std across 10 seeds) for MLP tag-based classifiers taking as input tags of different types: abstract, concrete and combined, with a varying number of tags used to describe an image. The performance is reported on three datasets: PrivacyAlert with subjective annotation, VISPR with object-guided annotation, and DIPA2 with subjective object-guided annotation. The presented F1-macro ranges vary in scale for each dataset.