Efficient Inference Using Large Language Models with Limited Human Data: Fine-Tuning then Rectification

TL;DR

Efficient Inference Using LLMs with Limited Human Data tackles estimating population quantities with limited labeled data by coupling fine-tuning and post-hoc rectification (Prediction-Powered Inference, PPI). The key idea is to train the LLM to minimize residual variance on the labeled data, which optimizes the downstream variance of the PPI estimator, and to use a scaling-law based rule to optimally allocate labeled samples between fine-tuning and rectification. The framework extends to general M-estimation and provides inference procedures, while empirical results on Wine Reviews show substantial labeling savings (about 45-66%) and superior variance reduction compared to baselines, validating both the method and the allocation rule. Together, these contributions offer a practical and theoretically grounded path for reliable, cost-efficient decision making with LLM surrogates in business contexts.

Abstract

Driven by recent advances in artificial intelligence (AI), a growing literature has demonstrated the potential for using large language models (LLMs) as scalable surrogates to generate human-like responses in many business applications. Two common approaches to improve the performance of LLMs include: fine-tuning, which aligns LLMs more closely with human responses, and rectification, which corrects biases in LLM outputs. In this paper, we develop a two-stage framework that combines fine-tuning and rectification, and optimally allocates limited labeled samples across the two stages. Unlike the conventional objective that minimizes the mean squared prediction errors, we propose to minimize the variance of the prediction errors as the fine-tuning objective, which is optimal for the downstream rectification stage. Building on this insight, we leverage the scaling law of fine-tuning to optimally allocate the limited labeled human data between the fine-tuning and rectification stages. Our empirical analysis validates the fine-tuning scaling law and confirms that our proposed optimal allocation rule reliably identifies the optimal sample allocation. We demonstrate substantial efficiency gains in estimation and inference performance relative to fine-tuning or rectification alone, or to employing the standard mean-squared error objective within the fine-tuning then rectification framework, resulting in significant cost savings for reliable business decisions.

Figures (6)

• Figure 1: Empirical validation of the variance-based scaling law. The estimated parameters are $\widehat{a} = 10.21$, $\widehat{\alpha} = 0.21$, and $\widehat{b} = 1.98$
• Figure 2: Performance of FT+PPI estimators under different sample allocations
• Figure 3: Efficiency gain ($R^2(s) - s/n$) from fine-tuning allocation. Positive values indicate regions where FT+PPI outperforms the Sample Mean Estimator.
• Figure EC.1: Bootstrap robustness of the scaling law fit estimated with $n=5{,}000$ labeled samples. Light curves correspond to individual bootstrap replicates, the solid line denotes the median fit, and the shaded region indicates the central $95\%$ robustness interval.
• Figure EC.2: Few-shot prompt used to obtain LLM surrogates for the PPI-Only estimator in the Wine Reviews study.

Theorems & Definitions (13)

• Example 1: Fine-tuning a Biased Predictor
• Remark 1
• Theorem 1: Optimal Allocation Rule
• Proposition 1: Characterization of the Optimal Allocation Rule
• Proposition 2: Criterion for Variance Reduction
• Proposition 3: Conditions for Variance Reduction under Scaling Laws
• Example 2: Applying Proposition \ref{['prop:variance_reduction_conditions']} to Evaluate FT+PPI
• Example 3: Mean Estimation
• Example 4: Categorical Response Estimation
• Example 5: Linear Regression