Detecting Content Rating Violations in Android Applications: A Vision-Language Approach

TL;DR

This work tackles automatic detection of Android app content-rating violations by introducing a vision-language model that jointly encodes app visuals (icons and screenshots) and textual descriptions. The architecture splits visual processing into content and style encoders, augments a text backbone, and uses cross-attention to fuse modalities, trained with a Sigmoid Contrastive Loss alongside a mean-squared-error style loss. On a large Google Play gaming dataset, the method outperforms CLIP-based baselines, achieving about a 5.9% relative accuracy boost, and successfully identifies dozens of potential malpractices in the wild, including some Teacher Approved apps. The results illustrate practical utility for regulators and app stores by surfacing misalignment signals that correlate with removals, while also revealing unverifiable or disguised content and providing actionable targets for inspection and policy enforcement.

Abstract

Despite regulatory efforts to establish reliable content-rating guidelines for mobile apps, the process of assigning content ratings in the Google Play Store remains self-regulated by the app developers. There is no straightforward method of verifying developer-assigned content ratings manually due to the overwhelming scale or automatically due to the challenging problem of interpreting textual and visual data and correlating them with content ratings. We propose and evaluate a visionlanguage approach to predict the content ratings of mobile game applications and detect content rating violations, using a dataset of metadata of popular Android games. Our method achieves ~6% better relative accuracy compared to the state-of-the-art CLIP-fine-tuned model in a multi-modal setting. Applying our classifier in the wild, we detected more than 70 possible cases of content rating violations, including nine instances with the 'Teacher Approved' badge. Additionally, our findings indicate that 34.5% of the apps identified by our classifier as violating content ratings were removed from the Play Store. In contrast, the removal rate for correctly classified apps was only 27%. This discrepancy highlights the practical effectiveness of our classifier in identifying apps that are likely to be removed based on user complaints.

Figures (8)

• Figure 1: (a): Vision-language model architecture during the training stage. (b): Custom transformer block with cross-attention. (c) Pipeline for the downstream task of content rating classification.
• Figure 2: Disparity between the content and style of app icons and screenshots.
• Figure 3: Examples belonging to 1) potential malpractices, and 2) potential disguises. For each app, the image on the left represents the app icon, and on the right is a screenshot. Red * represents app that removed from the Play store after the initial data crawl in 2023.
• Figure 4: Confusion matrix comparing our method against baselines - using image-text embeddings.
• Figure 5: Visualisation of image patches attending to text tokens in the custom cross-attention layer.