Mind Your Questions! Towards Backdoor Attacks on Text-to-Visualization Models

TL;DR

This study identifies and quantifies security vulnerabilities in text-to-visualization models by introducing VisPoison, a backdoor framework that employs stealthy triggers and payloads to cause data leakage, visualization errors, or DoS. It demonstrates the framework against both trainable and in-context learning (ICL) models on nvBench, achieving attack success rates above $90\%$ across six victim models. The work shows that existing defenses, including Onion-style perplexity filtering and semantic-change detectors, are largely ineffective against the dual-trigger strategy (rare-word and first-word) and payloads. Overall, VisPoison underscores an urgent need for robust, defense-ready text-to-vis systems and provides a blueprint for evaluating and strengthening model security in this domain.

Abstract

Text-to-visualization (text-to-vis) models have become valuable tools in the era of big data, enabling users to generate data visualizations and make informed decisions through natural language queries (NLQs). Despite their widespread application, the security vulnerabilities of these models have been largely overlooked. To address this gap, we propose VisPoison, a novel framework designed to identify these vulnerabilities of current text-to-vis models systematically. VisPoison introduces two types of triggers that activate three distinct backdoor attacks, potentially leading to data exposure, misleading visualizations, or denial-of-service (DoS) incidents. The framework features both proactive and passive attack mechanisms: proactive attacks leverage rare-word triggers to access confidential data, while passive attacks, triggered unintentionally by users, exploit a first-word trigger method, causing errors or DoS events in visualizations. Through extensive experiments on both trainable and in-context learning (ICL)-based text-to-vis models, \textit{VisPoison} achieves attack success rates of over 90\%, highlighting the security problem of current text-to-vis models. Additionally, we explore two types of defense mechanisms against these attacks, but the results show that existing countermeasures are insufficient, underscoring the pressing need for more robust security solutions in text-to-vis systems.

Figures (4)

• Figure 1: The workflow of VisPoison. The left blocks illustrate the three types of attacks designed in VisPoison, displaying the process of transforming clean examples into poisoned ones by inserting triggers and payloads. Red text indicates the specific triggers and payloads. The middle block represents the victim text-to-vis models. The right block shows the original visualization compared to the attacked visualization results.
• Figure 2: Backdoor attack performance for trainable text-to-vis models by VisPoison on the test set of nvBench with different poisoning rates.
• Figure 3: Backdoor attack performance for ICL-based text-to-vis model by VisPoison on the test set of nvBench with different poisoning rates.
• Figure 4: Defensive results of Onion to VisPoison.