Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cost-Aware Model Selection for Text Classification: Multi-Objective Trade-offs Between Fine-Tuned Encoders and LLM Prompting in Production

Alberto Andres Valdes Gonzalez

TL;DR

This work tackles cost-aware model selection for fixed-label text classification by systematically comparing fine-tuned encoder architectures against zero-/few-shot prompting of large language models across IMDB, SST-2, AG News, and DBPedia. It treats deployment as a multi-objective problem, jointly evaluating predictive accuracy, latency, and monetary cost, and uses a generalization-aware checkpointing criterion together with Pareto-frontier analyses and a latency-tolerant utility function $U(F1,Cost,Latency) = \frac{F1}{Cost} \cdot \exp(-\frac{Latency_{50}}{\tau})$ to guide decisions. Across all benchmarks, fine-tuned encoders consistently deliver competitive or superior accuracy with an order-of-magnitude lower latency and cost, highlighting the limitations of indiscriminate LLM use for standard classification workloads. The findings advocate a production-oriented paradigm where encoders serve as efficient, auditable decision engines, while LLMs play complementary roles in knowledge generation or hybrid architectures, with open artifacts and reproducible protocols enabling cost-aware, sustainable NLP deployment.

Abstract

Large language models (LLMs) such as GPT-4o and Claude Sonnet 4.5 have demonstrated strong capabilities in open-ended reasoning and generative language tasks, leading to their widespread adoption across a broad range of NLP applications. However, for structured text classification problems with fixed label spaces, model selection is often driven by predictive performance alone, overlooking operational constraints encountered in production systems. In this work, we present a systematic comparison of two contrasting paradigms for text classification: zero- and few-shot prompt-based large language models, and fully fine-tuned encoder-only architectures. We evaluate these approaches across four canonical benchmarks (IMDB, SST-2, AG News, and DBPedia), measuring predictive quality (macro F1), inference latency, and monetary cost. We frame model evaluation as a multi-objective decision problem and analyze trade-offs using Pareto frontier projections and a parameterized utility function reflecting different deployment regimes. Our results show that fine-tuned encoder-based models from the BERT family achieve competitive, and often superior, classification performance while operating at one to two orders of magnitude lower cost and latency compared to zero- and few-shot LLM prompting. Overall, our findings suggest that indiscriminate use of large language models for standard text classification workloads can lead to suboptimal system-level outcomes. Instead, fine-tuned encoders emerge as robust and efficient components for structured NLP pipelines, while LLMs are better positioned as complementary elements within hybrid architectures. We release all code, datasets, and evaluation protocols to support reproducibility and cost-aware NLP system design.

Cost-Aware Model Selection for Text Classification: Multi-Objective Trade-offs Between Fine-Tuned Encoders and LLM Prompting in Production

TL;DR

This work tackles cost-aware model selection for fixed-label text classification by systematically comparing fine-tuned encoder architectures against zero-/few-shot prompting of large language models across IMDB, SST-2, AG News, and DBPedia. It treats deployment as a multi-objective problem, jointly evaluating predictive accuracy, latency, and monetary cost, and uses a generalization-aware checkpointing criterion together with Pareto-frontier analyses and a latency-tolerant utility function to guide decisions. Across all benchmarks, fine-tuned encoders consistently deliver competitive or superior accuracy with an order-of-magnitude lower latency and cost, highlighting the limitations of indiscriminate LLM use for standard classification workloads. The findings advocate a production-oriented paradigm where encoders serve as efficient, auditable decision engines, while LLMs play complementary roles in knowledge generation or hybrid architectures, with open artifacts and reproducible protocols enabling cost-aware, sustainable NLP deployment.

Abstract

Large language models (LLMs) such as GPT-4o and Claude Sonnet 4.5 have demonstrated strong capabilities in open-ended reasoning and generative language tasks, leading to their widespread adoption across a broad range of NLP applications. However, for structured text classification problems with fixed label spaces, model selection is often driven by predictive performance alone, overlooking operational constraints encountered in production systems. In this work, we present a systematic comparison of two contrasting paradigms for text classification: zero- and few-shot prompt-based large language models, and fully fine-tuned encoder-only architectures. We evaluate these approaches across four canonical benchmarks (IMDB, SST-2, AG News, and DBPedia), measuring predictive quality (macro F1), inference latency, and monetary cost. We frame model evaluation as a multi-objective decision problem and analyze trade-offs using Pareto frontier projections and a parameterized utility function reflecting different deployment regimes. Our results show that fine-tuned encoder-based models from the BERT family achieve competitive, and often superior, classification performance while operating at one to two orders of magnitude lower cost and latency compared to zero- and few-shot LLM prompting. Overall, our findings suggest that indiscriminate use of large language models for standard text classification workloads can lead to suboptimal system-level outcomes. Instead, fine-tuned encoders emerge as robust and efficient components for structured NLP pipelines, while LLMs are better positioned as complementary elements within hybrid architectures. We release all code, datasets, and evaluation protocols to support reproducibility and cost-aware NLP system design.
Paper Structure (78 sections, 7 equations, 4 figures, 13 tables)

This paper contains 78 sections, 7 equations, 4 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Model Selection as a Knowledge-Based Process
  3. Related Work and Contribution
  4. Knowledge Artifacts for Operational Decision-Making
  5. Method: Fine-Tuned Encoders vs. Zero-/Few-Shot LLM Prompting
  6. Operational Assumptions and Decision Variables
  7. Reproducibility as a Knowledge Constraint
  8. Tasks and Datasets
  9. Model Families
  10. Encoders (fine-tuning).
  11. LLMs (prompting).
  12. Experimental Environment and Deployment Setup
  13. Training and Inference Protocol
  14. Fine-tuning schedule.
  15. Generalization-aware selection metric.
  16. ...and 63 more sections

Figures (4)

  • Figure 1: Pareto projections for IMDB sentiment classification. Each point represents a model configuration; colors denote model families and marker shapes indicate training or prompting setup.
  • Figure 2: Pareto projections for SST-2 sentiment classification under cost and latency constraints.
  • Figure 3: Pareto projections for AG News topic classification, illustrating accuracy–efficiency trade-offs in multi-class settings.
  • Figure 4: Pareto projections for DBPedia ontology classification. Encoder-based models saturate performance while remaining dominant in cost and latency.