Evaluating the Quality of Randomness and Entropy in Tasks Supported by Large Language Models

TL;DR

The paper addresses how large language models handle randomness in tasks ranging from direct random-number generation to shuffling and password creation. It introduces a benchmarking suite and a reproducibility framework, examining factors such as external tool access, task type, model state, and prompting, and evaluates outputs with entropy measures and the NIST randomness test-suite. Key findings show that LLMs can mimic randomness to some extent but exhibit significant inconsistencies and biases, with local PRNGs often outperforming LLM-generated randomness; external tooling can improve outcomes in some cases. The work provides practical guidance on diagnosing and improving randomness in LLM-based systems, highlighting security and fairness implications and outlining concrete steps for robust, reproducible assessments.

Abstract

The rapid advancement of large language model (LLM) technology has led to diverse applications, many of which inherently require randomness, such as stochastic decision-making, gaming, scheduling, AI agents, and cryptography-related tasks. However, the capabilities of LLMs in handling randomness, particularly in generating and utilizing random numbers effectively, remain unclear. This paper investigates the capacity of LLMs for handling tasks that involve randomness through a series of experiments. We designed a set of experiments that consider various factors that can influence an LLM's performance in tasks involving randomness, such as accessibility to external tools, types of tasks, model states (fresh vs. non-fresh), and prompting strategies. The experiments cover a range of tasks, including generating random numbers, generating random strings such as passwords, shuffling items, and evaluating the quality of randomness using entropy and the NIST randomness test-suite. Our findings reveal that while LLMs can generate outputs that exhibit some degree of randomness, their performance is inconsistent and often deviates significantly from the expected behavior. The analysis of the experimental results highlights key limitations and areas where improvement is needed for the LLMs to effectively handle tasks involving randomness

Figures (5)

• Figure 1: Gemini 1.5 Pro distribution of random numbers.
• Figure 2: Mistral Large(2047) distribution of random numbers.
• Figure 3: Gemini 1.5 with function calling (10k).
• Figure 4: LLM function call workflow.
• Figure 5: Gemini 1.5 with function calling (100k).