Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Security Vulnerabilities of Text-to-SQL Models

Xutan Peng, Yipeng Zhang, Jingfeng Yang, Mark Stevenson

TL;DR

The paper presents a security-focused evaluation of Text-to-SQL systems, showing that six commercial Text-to-SQL services are vulnerable to black-box injections that can cause information disclosure, tampering, and DoS, while four open-source Text-to-SQL models can be backdoored via poisoned training data with negligible impact on normal performance. It demonstrates that both real-world deployments and supply-chain pathways can be exploited, including recovery of database parameters and disruptive attacks, and shows that backdoors generalize across database schemata. The authors propose practical mitigations, such as input filtering, supplier verification, and defense-in-depth strategies, and advocate for automated vulnerability detection and responsible disclosure. The findings highlight a critical need for security-aware design and testing of NLP-based database interfaces as NLP adoption in production grows.

Abstract

Although it has been demonstrated that Natural Language Processing (NLP) algorithms are vulnerable to deliberate attacks, the question of whether such weaknesses can lead to software security threats is under-explored. To bridge this gap, we conducted vulnerability tests on Text-to-SQL systems that are commonly used to create natural language interfaces to databases. We showed that the Text-to-SQL modules within six commercial applications can be manipulated to produce malicious code, potentially leading to data breaches and Denial of Service attacks. This is the first demonstration that NLP models can be exploited as attack vectors in the wild. In addition, experiments using four open-source language models verified that straightforward backdoor attacks on Text-to-SQL systems achieve a 100% success rate without affecting their performance. The aim of this work is to draw the community's attention to potential software security issues associated with NLP algorithms and encourage exploration of methods to mitigate against them.

On the Security Vulnerabilities of Text-to-SQL Models

TL;DR

The paper presents a security-focused evaluation of Text-to-SQL systems, showing that six commercial Text-to-SQL services are vulnerable to black-box injections that can cause information disclosure, tampering, and DoS, while four open-source Text-to-SQL models can be backdoored via poisoned training data with negligible impact on normal performance. It demonstrates that both real-world deployments and supply-chain pathways can be exploited, including recovery of database parameters and disruptive attacks, and shows that backdoors generalize across database schemata. The authors propose practical mitigations, such as input filtering, supplier verification, and defense-in-depth strategies, and advocate for automated vulnerability detection and responsible disclosure. The findings highlight a critical need for security-aware design and testing of NLP-based database interfaces as NLP adoption in production grows.

Abstract

Although it has been demonstrated that Natural Language Processing (NLP) algorithms are vulnerable to deliberate attacks, the question of whether such weaknesses can lead to software security threats is under-explored. To bridge this gap, we conducted vulnerability tests on Text-to-SQL systems that are commonly used to create natural language interfaces to databases. We showed that the Text-to-SQL modules within six commercial applications can be manipulated to produce malicious code, potentially leading to data breaches and Denial of Service attacks. This is the first demonstration that NLP models can be exploited as attack vectors in the wild. In addition, experiments using four open-source language models verified that straightforward backdoor attacks on Text-to-SQL systems achieve a 100% success rate without affecting their performance. The aim of this work is to draw the community's attention to potential software security issues associated with NLP algorithms and encourage exploration of methods to mitigate against them.
Paper Structure (28 sections, 11 equations, 5 figures, 3 tables)

This paper contains 28 sections, 11 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Large Language Models
  4. Text-to-SQL Algorithms
  5. (Un)Reliability of Code Generation
  6. Attacking NLP Models
  7. Preliminaries: Top Security Threats
  8. Information Disclosure
  9. Tampering
  10. Denial of Service (DoS)
  11. Methodology
  12. Black-Box Attacks by End User
  13. In-Band Injection
  14. Blind Injection
  15. Backdoor Attacks by Model Supplier
  16. ...and 13 more sections

Figures (5)

  • Figure 1: Two positive vulnerability tests on Baidu-UNIT through its Text-to-SQL module. "单位是...的巫师有哪些" in the Chinese questions means "Which wizard's affiliation is ..." in English (also in Fig \ref{['fig:baidu_results']}). See \ref{['sssec:exp-wild-baidu']} for details.
  • Figure 2: Illustration of black-box attacks by the End User.
  • Figure 3: Data table frequently used by examples in \ref{['sec:method']} and \ref{['sec:exp']}.
  • Figure 4: Illustration of backdoor attacks (via data poisoning) by the Model Supplier. There are $t$ samples in the clean fine-tuning data set.
  • Figure 5: Screenshots of Baidu-UNIT's browser-based bot during vulnerability tests using the blind injection strategy (see \ref{['sssec:blind-inject']}).