RECALL-MM: A Multimodal Dataset of Consumer Product Recalls for Risk Analysis using Computational Methods and Large Language Models

TL;DR

This work tackles the underutilization of product recall data for risk analysis in engineering design by constructing RECALL-MM, a multimodal CPSC recall dataset augmented with LLM-generated textual and visual descriptors and classifications. It establishes a processing pipeline that embeddings recall descriptions and product names with Sentence-BERT, visualizes them via tsne, and tests an LLM-based hazard predictor from image descriptions. Through three case studies, the authors demonstrate how recall-space embeddings and image-driven hazard prediction can reveal hazard patterns, contextualize new product ideas, and identify limitations where hazards are non-visual. The dataset and methods offer a scalable, data-driven pathway to proactive risk mitigation in design and pave the way for integrating recall-informed insights into early-stage requirements and cross-national safety analyses.

Abstract

Product recalls provide valuable insights into potential risks and hazards within the engineering design process, yet their full potential remains underutilized. In this study, we curate data from the United States Consumer Product Safety Commission (CPSC) recalls database to develop a multimodal dataset, RECALL-MM, that informs data-driven risk assessment using historical information, and augment it using generative methods. Patterns in the dataset highlight specific areas where improved safety measures could have significant impact. We extend our analysis by demonstrating interactive clustering maps that embed all recalls into a shared latent space based on recall descriptions and product names. Leveraging these data-driven tools, we explore three case studies to demonstrate the dataset's utility in identifying product risks and guiding safer design decisions. The first two case studies illustrate how designers can visualize patterns across recalled products and situate new product ideas within the broader recall landscape to proactively anticipate hazards. In the third case study, we extend our approach by employing a large language model (LLM) to predict potential hazards based solely on product images. This demonstrates the model's ability to leverage visual context to identify risk factors, revealing strong alignment with historical recall data across many hazard categories. However, the analysis also highlights areas where hazard prediction remains challenging, underscoring the importance of risk awareness throughout the design process. Collectively, this work aims to bridge the gap between historical recall data and future product safety, presenting a scalable, data-driven approach to safer engineering design.

Figures (7)

• Figure 1: Process overview of translating database information into nine distinct data fields.
• Figure 2: Data metrics from 6,874 recalls spanning 2000 - 2024 recall dates.
• Figure 3: Correlation matrix of hazards and product classifications.
• Figure 4: Embedding space of recall descriptions labeled by product categories.
• Figure 5: a) Embedded recall descriptions of electrical (black) and clothing_accessories (gray). b) Embedded recall descriptions of baby_products (blue) and sports_recreation (purple). Descriptions are denoted for each example, showcasing distant (a) and near (b) recall descriptions across different product categories.