Enabling Fine-Grained Operating Points for Black-Box LLMs

TL;DR

This work tackles the challenge of achieving fine-grained operating points when using black-box LLMs for decision making under constraints. By analyzing the rounding bias in verbally expressed probabilities, it shows that standard prompting yields low cardinality and coarse ROC/PR curves. The authors propose noise-augmented and supervised mapping approaches to substantially increase the density of operating points while maintaining or improving performance across 11 datasets and 3 LLMs. Their results enable more precise constraint-based deployment of black-box LLMs, reducing the need for model fine-tuning and extensive sampling, with potential extensions through probabilistic priors in generative models.

Abstract

Black-box Large Language Models (LLMs) provide practical and accessible alternatives to other machine learning methods, as they require minimal labeled data and machine learning expertise to develop solutions for various decision making problems. However, for applications that need operating with constraints on specific metrics (e.g., precision $\geq$ 95%), decision making with black-box LLMs remains unfavorable, due to their low numerical output cardinalities. This results in limited control over their operating points, preventing fine-grained adjustment of their decision making behavior. In this paper, we study using black-box LLMs as classifiers, focusing on efficiently improving their operational granularity without performance loss. Specifically, we first investigate the reasons behind their low-cardinality numerical outputs and show that they are biased towards generating rounded but informative verbalized probabilities. Then, we experiment with standard prompt engineering, uncertainty estimation and confidence elicitation techniques, and observe that they do not effectively improve operational granularity without sacrificing performance or increasing inference cost. Finally, we propose efficient approaches to significantly increase the number and diversity of available operating points. Our proposed approaches provide finer-grained operating points and achieve comparable to or better performance than the benchmark methods across 11 datasets and 3 LLMs.

Figures (17)

• Figure 1: PR and ROC curves of 3 LLMs' verbalized probability estimates (vrb) together with Qwen's decision token probabilities (tkn) across 11 datasets. Histograms of verbalized probabilities over individual axes are also provided to visualize the sparsity along these axes. All three LLMs' verbalized probability distributions imply low cardinality and diversity of operating points, while using token probabilities provides a highly continuous curve (its histogram is omitted for brevity).
• Figure 2: All 3 LLMs are heavily biased towards verbalizing probability estimates that end with "0" and "5"s. On the other hand, the distributions of the 3rd characters are spread across all tokens (digits) with slights bias towards extremes. Interestingly we observe that even though all three models' top 2 most frequent tokens in position 4 are consistent, Qwen's highest probability token is "5" while the others' is "0".
• Figure 3: Verbalized probability distributions of Qwen using prompts with low, medium and high specificity. The output cardinality can be increased by providing additional instructions on how to conduct the task.
• Figure 4: PRAUC of different approaches over $5$ different train/test splits with $2\sigma$ error bars. Proposed methods are shown in purple. (Top) Aggregated results over 11 datasets. While naive prompting is usually worse than other methods, no method consistently outperforms the rest. Since a majority of these datasets are small and have nearly monotonic PR curves, the proposed supervised methods do not provide significant boosts. (Bottom) Results on combined dataset. Supervised approaches vastly outperform the rest including the sampling approaches requiring $20$ LLM calls per instance.
• Figure 5: Scatter plot of precision-recall values for all possible thresholds. The plots along the margin show the histogram of operating points for each method. The proposed methods have significantly higher output cardinality, providing better control over the operating point, while simultaneously improving the performance.