Mass-Producing Failures of Multimodal Systems with Language Models

TL;DR

The paper presents MultiMon, an autonomous evaluation pipeline for multimodal systems that detects systematic and individual failures by harvesting erroneous agreement via CLIP embeddings and then uses large language models to categorize and generate new failure instances. It demonstrates 14 high-quality systematic failures of the CLIP text encoder and shows that these failures transfer to downstream text-to-image, text-to-3D, and text-to-video models, with substantial downstream error rates. The approach supports steering towards subdomain applications (e.g., self-driving) and can reveal safety-filter vulnerabilities, illustrating both the potential and risks of scalable automatic evaluation. Overall, MultiMon offers a simple, generalizable framework for broad, model-agnostic failure discovery that scales with advancing language models and embedding techniques, forming a foundation for robust evaluation pipelines in multimodal AI.

Abstract

Deployed multimodal systems can fail in ways that evaluators did not anticipate. In order to find these failures before deployment, we introduce MultiMon, a system that automatically identifies systematic failures -- generalizable, natural-language descriptions of patterns of model failures. To uncover systematic failures, MultiMon scrapes a corpus for examples of erroneous agreement: inputs that produce the same output, but should not. It then prompts a language model (e.g., GPT-4) to find systematic patterns of failure and describe them in natural language. We use MultiMon to find 14 systematic failures (e.g., "ignores quantifiers") of the CLIP text-encoder, each comprising hundreds of distinct inputs (e.g., "a shelf with a few/many books"). Because CLIP is the backbone for most state-of-the-art multimodal systems, these inputs produce failures in Midjourney 5.1, DALL-E, VideoFusion, and others. MultiMon can also steer towards failures relevant to specific use cases, such as self-driving cars. We see MultiMon as a step towards evaluation that autonomously explores the long tail of potential system failures. Code for MULTIMON is available at https://github.com/tsb0601/MultiMon.

Figures (15)

• Figure 1: Examples failures that MultiMon generates on state-of-the-art text-to-image systems.
• Figure 2: The MultiMon pipeline. Left.MultiMon starts with a corpus of sentences (dots), then identifies individual failures: pairs that have similar CLIP embeddings but should not (circled red dots). Center.MultiMon takes the individual failures, then categorizes them into systematic failures using a language model. Right.MultiMon takes the systematic failures, then generates new individual failures from them using a language model, which then generate incorrect images.
• Figure 3: We report whether each LM-corpus pair uncovers each systematic failure (checkmark), along with the success rate. Both the language model and corpus influence the systematic failures that MultiMon uncovers. We include raw success rates and error bars in Appendix \ref{['appendix:ablation_corpus']}.
• Figure 4: Examples of inputs that MultiMon generates. Since MultiMon uses CLIP to find failures, a single input produces the same error in many state-of-the-art text-to-image systems.
• Figure 5: Top. Example of a 3d-scene Shape-E generates with 8 chairs instead of 4, rotated at different angles. Bottom. Example of a video VideoFusion generates of a wind turbine spinning, instead of at rest, captured at different frames.