Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenization

TL;DR

This work highlights tokenization as a fundamental bottleneck limiting LLM accuracy, especially for Chinese input. It introduces the Adversarial Dataset for Tokenizer (ADT), comprising manually crafted ADT-Human and automatically generated ADT-Auto to stress-test tokenization across diverse models. Empirical results show tokenization errors strongly correlate with incorrect outputs across both open- and closed-source models, including GPT-4o and Llama-3, indicating tokenization defenses are essential for robustness. The authors provide vocabulary-export and automatic data-generation pipelines, offering a practical framework to probe and improve tokenization-based vulnerabilities in large language systems. The findings underscore the need to integrate tokenization-aware strategies into model design and evaluation to enhance reliability in real-world use cases, particularly for non-English languages.

Abstract

Large Language Models (LLMs) have shown remarkable capabilities in language understanding and generation. Nonetheless, it was also witnessed that LLMs tend to produce inaccurate responses to specific queries. This deficiency can be traced to the tokenization step LLMs must undergo, which is an inevitable limitation inherent to all LLMs. In fact, incorrect tokenization is the critical point that hinders LLMs in understanding the input precisely, thus leading to unsatisfactory output. This defect is more obvious in Chinese scenarios. To demonstrate this flaw of LLMs, we construct an adversarial dataset, named as $\textbf{ADT (Adversarial Dataset for Tokenizer)}$, which draws upon the vocabularies of various open-source LLMs to challenge LLMs' tokenization. ADT consists of two subsets: the manually constructed ADT-Human and the automatically generated ADT-Auto. Our empirical results reveal that our ADT is highly effective on challenging the tokenization of leading LLMs, including GPT-4o, Llama-3, Deepseek-R1 and so on, thus degrading these LLMs' capabilities. Moreover, our method of automatic data generation has been proven efficient and robust, which can be applied to any open-source LLMs. In this paper, we substantially investigate LLMs' vulnerability in terms of challenging their token segmentation, which will shed light on the subsequent research of improving LLMs' capabilities through optimizing their tokenization process and algorithms.

Figures (22)

• Figure 1: Two instances of LLM generating incorrect response due to incorrect tokenization. Case 1 is a Chinese input instance, of which the English translation is noted below according to its correct tokenization. In Case 2, a space is omitted between 'sing' and 'lemon', causing the LLM's incorrect tokenization, which is detailed in Section \ref{['Construction']}.
• Figure 2: Our framework of constructing ADT-Human manually.
• Figure 3: Our framework of generating ADT-Auto automatically.
• Figure 4: Four relationships between tokenization and response, take Qwen-7B-Chat as an example.
• Figure :