D-Models and E-Models: Diversity-Stability Trade-offs in the Sampling Behavior of Large Language Models

TL;DR

This work investigates how large language models sample tokens by relating the token-level distribution $P_{token}$ to the task-level distribution $P_{task}$ and the resulting distribution $P_{result}$. It identifies two distinct model types, D-models (deterministic, highly concentrated $P_{token}$) and E-models (exploratory, $P_{token}$ aligned with $P_{task}$), and validates them through simulated distribution tasks and downstream applications. The study reveals a diversity–stability trade-off: D-models excel in deterministic, syntax-sensitive tasks like code generation but may diverge from task constraints, while E-models better align with $P_{task}$ in exploratory tasks like recommendations, offering more stable performance under uncertainty. These findings inform model selection and sampling configuration for web-scale applications, enabling a balance between diversity and reliability in real-world uncertainty. The work contributes a probabilistic framework for understanding web-task sampling, formal definitions of D-/E-models, empirical evidence of their behaviors, and insights into internal mechanisms (temperature effects, layer dynamics, and prior analyses) that can guide practical deployment. Overall, the paper provides a diagnostic lens to interpret LLM sampling, highlighting when to favor deterministic versus exploratory models to optimize performance across code, search, and recommendation tasks in realistic settings.

Abstract

The predictive probability of the next token (P_token) in large language models (LLMs) is inextricably linked to the probability of relevance for the next piece of information, the purchase probability of the next product, and the execution probability of the next action-all of which fall under the scope of the task-level target distribution (P_task). While LLMs are known to generate samples that approximate real-world distributions, whether their fine-grained sampling probabilities faithfully align with task requirements remains an open question. Through controlled distribution-sampling simulations, we uncover a striking dichotomy in LLM behavior, distinguishing two model types: D-models (e.g. Qwen-2.5), whose P_token exhibits large step-to-step variability and poor alignment with P_task; and E-models (e.g. Mistral-Small), whose P_token is more stable and better aligned with P_task. We further evaluate these two model types in downstream tasks such as code generation and recommendation, revealing systematic trade-offs between diversity and stability that shape task outcomes. Finally, we analyze the internal properties of both model families to probe their underlying mechanisms. These findings offer foundational insights into the probabilistic sampling behavior of LLMs and provide practical guidance on when to favor D- versus E-models. For web-scale applications, including recommendation, search, and conversational agents, our results inform model selection and configuration to balance diversity with reliability under real-world uncertainty, providing a better level of interpretation.

Figures (10)

• Figure 1: We analyze the probabilistic distribution sampling capabilities of LLMs by combining $P_\text{task}$, $P_\text{token}$, and $P_\text{result}$, and have verified the existence of both the D-model and the E-model. Although there are minor discrepancies in the $P_\text{result}$, which aligns somewhat with the $P_\text{task}$, they demonstrate distinct patterns at the fine-grained $P_\text{token}$ level. We have also analyzed both models within downstream task scenarios and their internal characteristics.
• Figure 2: Comparison of $P_\text{token}$ between D- and E-model for distribution {1: 0.1, 2: 0.7, 3: 0.1, 4: 0.1}. The horizontal axis denotes the generation step, while the vertical axis represents the corresponding token probability distribution.
• Figure 3: The $P_\text{result}$ of various models. In the extreme task (top), $P_{\text{result}}(2)$ > $P_{\text{task}}(2) = 0.7$, whereas in the flat task (bottom), $P_{\text{result}}(4)$ > $P_{\text{task}}(4) = 0.2$ for most models.
• Figure 4: E-scores comparison of models, with error bars representing standard deviations.
• Figure 5: The distribution of maximum logits probabilities for 1,000 random problems in MMLU dataset, as demonstrated by D-model Qwen-2.5 and E-model Mistral-Small.