Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NICO-RAG: Multimodal Hypergraph Retrieval-Augmented Generation for Understanding the Nicotine Public Health Crisis

Manuel Serna-Aguilera, Raegan Anderes, Page Dobbs, Khoa Luu

TL;DR

The Nicotine Innovation Counter-Offensive Dataset is introduced to provide public health researchers with over 200,000 multimodal samples, including images and text descriptions, on 55 tobacco and nicotine product brands and NICO-RAG, a retrieval-augmented generation (RAG) framework that can retrieve image features without incurring the high-cost of language models, as well as the added cost of processing image tokens with large-scale datasets such as NICO.

Abstract

The nicotine addiction public health crisis continues to be pervasive. In this century alone, the tobacco industry has released and marketed new products in an aggressive effort to lure new and young customers for life. Such innovations and product development, namely flavored nicotine or tobacco such as nicotine pouches, have undone years of anti-tobacco campaign work. Past work is limited both in scope and in its ability to connect large-scale data points. Thus, we introduce the Nicotine Innovation Counter-Offensive (NICO) Dataset to provide public health researchers with over 200,000 multimodal samples, including images and text descriptions, on 55 tobacco and nicotine product brands. In addition, to provide public health researchers with factual connections across a large-scale dataset, we propose NICO-RAG, a retrieval-augmented generation (RAG) framework that can retrieve image features without incurring the high-cost of language models, as well as the added cost of processing image tokens with large-scale datasets such as NICO. At construction time, NICO-RAG organizes image- and text-extracted entities and relations into hypergraphs to produce as factual responses as possible. This joint multimodal knowledge representation enables NICO-RAG to retrieve images for query answering not only by visual similarity but also by the semantic similarity of image descriptions. Experimentals show that without needing to process additional tokens from images for over 100 questions, NICO-RAG performs comparably to the state-of-the-art RAG method adapted for images.

NICO-RAG: Multimodal Hypergraph Retrieval-Augmented Generation for Understanding the Nicotine Public Health Crisis

TL;DR

The Nicotine Innovation Counter-Offensive Dataset is introduced to provide public health researchers with over 200,000 multimodal samples, including images and text descriptions, on 55 tobacco and nicotine product brands and NICO-RAG, a retrieval-augmented generation (RAG) framework that can retrieve image features without incurring the high-cost of language models, as well as the added cost of processing image tokens with large-scale datasets such as NICO.

Abstract

The nicotine addiction public health crisis continues to be pervasive. In this century alone, the tobacco industry has released and marketed new products in an aggressive effort to lure new and young customers for life. Such innovations and product development, namely flavored nicotine or tobacco such as nicotine pouches, have undone years of anti-tobacco campaign work. Past work is limited both in scope and in its ability to connect large-scale data points. Thus, we introduce the Nicotine Innovation Counter-Offensive (NICO) Dataset to provide public health researchers with over 200,000 multimodal samples, including images and text descriptions, on 55 tobacco and nicotine product brands. In addition, to provide public health researchers with factual connections across a large-scale dataset, we propose NICO-RAG, a retrieval-augmented generation (RAG) framework that can retrieve image features without incurring the high-cost of language models, as well as the added cost of processing image tokens with large-scale datasets such as NICO. At construction time, NICO-RAG organizes image- and text-extracted entities and relations into hypergraphs to produce as factual responses as possible. This joint multimodal knowledge representation enables NICO-RAG to retrieve images for query answering not only by visual similarity but also by the semantic similarity of image descriptions. Experimentals show that without needing to process additional tokens from images for over 100 questions, NICO-RAG performs comparably to the state-of-the-art RAG method adapted for images.
Paper Structure (10 sections, 5 equations, 3 figures, 3 tables)

This paper contains 10 sections, 5 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. The Nicotine Innovation Counter-Offensive Dataset
  3. Nicotine Innovation Counter-Offensive Retrieval Augmented Generation (NICO-RAG) Approach
  4. Preliminaries
  5. Multimodal Knowledge Construction
  6. Multimodal Knowledge Retrieval for Question Answering
  7. Experiments and Results
  8. Experimental Design
  9. Experimental Results and Discussion
  10. Conclusion

Figures (3)

  • Figure 1: Samples of our NICO Dataset. (Left) Nicotine pouch samples with (a) mint-based flavors, (b) fruit flavors, and (c) spice and coffee flavors. (Right) Other tobacco and nicotine products with flavorings.
  • Figure 2: The NICO-RAG framework. We take in a query image and text, and via text and image entity discovery, we create the multimodal hypergraph knowledge $\mathcal{K}$, giving us image descriptors to capture all aspects desirable for public health in tobacco and nicotine prevention research.
  • Figure 3: Responses from NICO-RAG and a multimodal Hypergraph RAG luo2025hypergraphrag for two nicotine pouch products, where a complex search for inter- and intra-product information on image attributes is needed. Best viewed in zoom and in color. Green highlights denote correct descriptions, while red denotes lower-quality or incorrect descriptions.