Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Detecting Visual Triggers in Cannabis Imagery: A CLIP-Based Multi-Labeling Framework with Local-Global Aggregation

Linqi Lu, Xianshi Yu, Akhil Perumal Reddy

TL;DR

This paper tackles how visual triggers and textual framing in online cannabis edible content influence user engagement. It combines a CLIP-based multi-labeling framework with Local-Global Aggregation to identify food-related visuals in 42,743 Facebook images and uses BART-derived topic modeling to characterize post texts via ten STM-derived topics. The study finds positive associations between food-like visuals (e.g., candy, bakery, fruit) and engagement, and positive links between engagement and topics such as cannabis legalization, while colorfulness and certain topics show negative relationships. These findings offer actionable guidance for policymakers and regulators to craft warning labels and marketing guidelines to mitigate risks associated with cannabis edibles and youth exposure.

Abstract

This study investigates the interplay of visual and textual features in online discussions about cannabis edibles and their impact on user engagement. Leveraging the CLIP model, we analyzed 42,743 images from Facebook (March 1 to August 31, 2021), with a focus on detecting food-related visuals and examining the influence of image attributes such as colorfulness and brightness on user interaction. For textual analysis, we utilized the BART model as a denoising autoencoder to classify ten topics derived from structural topic modeling, exploring their relationship with user engagement. Linear regression analysis identified significant positive correlations between food-related visuals (e.g., fruit, candy, and bakery) and user engagement scores, as well as between engagement and text topics such as cannabis legalization. In contrast, negative associations were observed with image colorfulness and certain textual themes. These findings offer actionable insights for policymakers and regulatory bodies in designing warning labels and marketing regulations to address potential risks associated with recreational cannabis edibles.

Detecting Visual Triggers in Cannabis Imagery: A CLIP-Based Multi-Labeling Framework with Local-Global Aggregation

TL;DR

This paper tackles how visual triggers and textual framing in online cannabis edible content influence user engagement. It combines a CLIP-based multi-labeling framework with Local-Global Aggregation to identify food-related visuals in 42,743 Facebook images and uses BART-derived topic modeling to characterize post texts via ten STM-derived topics. The study finds positive associations between food-like visuals (e.g., candy, bakery, fruit) and engagement, and positive links between engagement and topics such as cannabis legalization, while colorfulness and certain topics show negative relationships. These findings offer actionable guidance for policymakers and regulators to craft warning labels and marketing guidelines to mitigate risks associated with cannabis edibles and youth exposure.

Abstract

This study investigates the interplay of visual and textual features in online discussions about cannabis edibles and their impact on user engagement. Leveraging the CLIP model, we analyzed 42,743 images from Facebook (March 1 to August 31, 2021), with a focus on detecting food-related visuals and examining the influence of image attributes such as colorfulness and brightness on user interaction. For textual analysis, we utilized the BART model as a denoising autoencoder to classify ten topics derived from structural topic modeling, exploring their relationship with user engagement. Linear regression analysis identified significant positive correlations between food-related visuals (e.g., fruit, candy, and bakery) and user engagement scores, as well as between engagement and text topics such as cannabis legalization. In contrast, negative associations were observed with image colorfulness and certain textual themes. These findings offer actionable insights for policymakers and regulatory bodies in designing warning labels and marketing regulations to address potential risks associated with recreational cannabis edibles.

Paper Structure

This paper contains 14 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Data Collection and Processing
  4. Engagement Score
  5. Data Summaries
  6. Extract Features From Images and Texts
  7. Colorfulness and Brightness
  8. Visual Object Detection
  9. Performance of CLIP Multi-Labeling
  10. Identify Topics in Post Texts
  11. Analyze association with online activities
  12. Quantification of Online Interactions.
  13. Preliminary Results
  14. Discussion and Conclusion

Figures (3)

  • Figure 1:
  • Figure 2:
  • Figure 3: Evaluation of CLIP model performance in image labeling. The figure presents the Area Under the Curve (AUC) scores, using manual annotations as the reference standard. The results compare the performance of the CLIP model with Local-Global Aggregation (LGA) scores (orange) against the baseline performance of the CLIP model without LGA (blue), where similarity scores for the entire image were thresholded directly.