DEFEND: A Large-scale 1M Dataset and Foundation Model for Tobacco Addiction Prevention

TL;DR

The paper tackles the mismatch between rapid tobacco product innovation and public health surveillance by introducing Tobacco-1M, a dataset of 1 million images across 75 tobacco product categories with four hierarchical label levels, and DEFEND, a foundation model for tobacco product understanding. DEFEND combines a dual-branch teacher-student framework, a Feature Enhancement Module, Local-Global Visual Coherence, and Enhanced Image-Text Alignment to learn rich multimodal representations that connect product visuals with textual descriptions. It demonstrates strong performance on product classification (83.1%), visual question answering (73.8%), and zero-shot generalization (45.6% on novel categories), outperforming several baselines and demonstrating regulatory-relevant capabilities. These results indicate DEFEND’s potential to support regulatory bodies and health researchers in monitoring emerging tobacco products and marketing strategies, thereby enhancing tobacco control and public health surveillance.

Abstract

While tobacco advertising innovates at unprecedented speed, traditional surveillance methods remain frozen in time, especially in the context of social media. The lack of large-scale, comprehensive datasets and sophisticated monitoring systems has created a widening gap between industry advancement and public health oversight. This paper addresses this critical challenge by introducing Tobacco-1M, a comprehensive dataset of one million tobacco product images with hierarchical labels spanning 75 product categories, and DEFEND, a novel foundation model for tobacco product understanding. Our approach integrates a Feature Enhancement Module for rich multimodal representation learning, a Local-Global Visual Coherence mechanism for detailed feature discrimination, and an Enhanced Image-Text Alignment strategy for precise product characterization. Experimental results demonstrate DEFEND's superior performance, achieving 83.1% accuracy in product classification and 73.8% in visual question-answering tasks, outperforming existing methods by significant margins. Moreover, the model exhibits robust zero-shot learning capabilities with 45.6% accuracy on novel product categories. This work provides regulatory bodies and public health researchers with powerful tools for monitoring emerging tobacco products and marketing strategies, potentially revolutionizing approaches to tobacco control and public health surveillance.

Figures (5)

• Figure 1: Examples of Our Tobacco-1M Dataset for Tobacco Product Understanding. The left section displays visual samples from the major product categories, including Combustible Products, Non-Combustible Products, and Nicotine Replacement Products. The right section demonstrates a hierarchical annotation example of an E-cigarette product with detailed categorical descriptions. Note: This dataset is not intended for the promotion of any tobacco products.
• Figure 2: The Distribution of Tobacco Product Categories (left) which illustrates the primary classification of our Tobacco-1M dataset with its four major domains, while The Distribution of Product Type Hierarchy (center) provides a detailed breakdown of product types and their associated brands. Word Cloud (right) of the most frequent product types in our dataset. Best viewed with zoom-in and color.
• Figure 3: Comparisons of Previous Methods. Prior works yu2022cocahe2022maskedfan2021multiscale9779951 demonstrate significant limitations in tobacco product analysis: naive feature fusion fails to capture nuanced product characteristics due to oversimplified integration, basic classification architecture overlooks critical regulatory elements. DEFEND overcomes these limitations through Feature Enhancement Module (FEM), enabling precise feature correlation across scales, that distinguishes critical regulatory elements and health impact indicators through targeted region focus and multi-scale attention mechanisms. Best viewed with zoom in and color.
• Figure 4: Overview of our proposed DEFEND Framework. We employ a dual-stream design where text input is processed through a Text Encoder while image input follows two parallel paths through a Teacher-Student architecture. The Feature Enhancement Module enhances these multimodal features, which are then used to train the model with multiple objectives, including contrastive, description, and patch coherence losses, to generate comprehensive multimodal representations.
• Figure 5: Attention Visualization. Compared to MAE he2022masked, our model demonstrates enhanced sensitivity to product-specific details in tobacco imagery. The model effectively highlights key product features and warning labels, maintaining robust attention even in challenging scenarios with varied backgrounds and user-generated content. Best viewed in color.