GlitchMiner: Mining Glitch Tokens in Large Language Models via Gradient-based Discrete Optimization

TL;DR

GlitchMiner addresses the reliability challenge posed by glitch tokens in large language models by reframing detection as a behavior-driven, entropy-maximization task. It introduces a gradient-guided local search that operates in a token embedding neighborhood to efficiently identify high-entropy, and thus potentially unstable, token candidates, followed by robust verification across multiple templates. Across 10 LLMs from five model families, GlitchMiner outperforms state-of-the-art baselines in Detected@N and shows strong efficiency gains, demonstrating both generality and practicality. This work provides a architecture-agnostic auditing tool for LLM safety and reliability with code available for adoption and further mitigation work for broader deployment.

Abstract

Glitch tokens, inputs that trigger unpredictable or anomalous behavior in Large Language Models (LLMs), pose significant challenges to model reliability and safety. Existing detection methods primarily rely on heuristic embedding patterns or statistical anomalies within internal representations, limiting their generalizability across different model architectures and potentially missing anomalies that deviate from observed patterns. We introduce GlitchMiner, an behavior-driven framework designed to identify glitch tokens by maximizing predictive entropy. Leveraging a gradient-guided local search strategy, GlitchMiner efficiently explores the discrete token space without relying on model-specific heuristics or large-batch sampling. Extensive experiments across ten LLMs from five major model families demonstrate that GlitchMiner consistently outperforms existing approaches in detection accuracy and query efficiency, providing a generalizable and scalable solution for effective glitch token discovery. Code is available at [https://github.com/wooozihu/GlitchMiner]

Figures (7)

• Figure 1: An illustrative example of how glitch token causing Llama2-7b-chat to fail a simple repetition task. More examples can be found in Appendix B.
• Figure 2: Visualization of GlitchMiner’s local search process. The pivot token (black, $t_c$) serves as the reference point. Its neighbor tokens (orange and red) represent the $K=4$ closest tokens in embedding space. Among these, the candidate batch tokens (red) are the top $B=2$ tokens with the highest approximate entropy values, estimated via first-order Taylor approximation. Tokens outside the neighborhood (gray) are excluded to maintain approximation accuracy and computational efficiency.
• Figure 3: Impact of different Neighborhood Size $K$ on GlitchMiner's performance
• Figure 4: Impact of different Batch Size $B$ on GlitchMiner's performance
• Figure 5: Effect of Initialization Method on GlitchMiner's Detected@1000 score.