Language Ranker: A Lightweight Ranking framework for LLM Decoding

TL;DR

Language Ranker reframes LLM decoding as a ranking problem, introducing a lightweight, feature-sharing ranker that reranks candidate responses produced by the base model. By extracting and projecting hidden features from a designated backbone layer, the ranker (listwise or pointwise) achieves performance comparable to large reward models with far fewer parameters and can run on CPUs for training and inference. Across math, coding, and function calling tasks, the approach delivers significant gains with minimal overhead and demonstrates strong cross-domain transfer, underscoring its practical potential for scalable, personalized LLM inference. The work argues that coupling a decoupled, efficient ranker with existing base models can substantially improve decoding effectiveness without substantial computational costs, enabling broader deployment and personalization.

Abstract

Conventional research on large language models (LLMs) has primarily focused on refining output distributions, while paying less attention to the decoding process that transforms these distributions into final responses. Recent advances, such as scaling the computation of inference time with reward models, have underscored the importance of decoding, but these methods often suffer from high computational costs and limited applicability. In this paper, we revisit LLM generation through the lens of recommender systems, conceptualizing the decoding process as analogous to the ranking stage in recommendation pipelines. From this perspective, we observe that both traditional decoding methods and reward models exhibit clear limitations such as redundancy. Motivated by this insight, we propose Language Ranker, a novel framework that introduces a lightweight module to rerank candidate responses using features extracted by the base model. Experiments across a wide range of tasks show that Language Ranker achieves performance comparable to large-scale reward models, while requiring only <0.5M additional parameters, significantly reducing the computational overhead during both training and inference stages. This highlights the efficiency and effectiveness of our method, showcasing its potential to fully unlock the capabilities of LLMs.

Figures (7)

• Figure 1: The comparison among existing methods, recommender system and our Language Ranker. The two charts on the left highlight the limitations of existing decoding strategies and reward models, in contrast to the recommender system pipeline shown in the top-right. Language Ranker addresses these limitations by incorporating a feature-shared, learnable, and lightweight ranker.
• Figure 2: The Framework of Language Ranker. The base model first generates multiple candidate responses. The hidden states corresponding to the final tokens of both the instruction and each candidate response are then extracted from a predefined layer and used as features. Finally, the ranker selects the most appropriate response based on these features.
• Figure 3: Personalized Language Ranker. We can pair a single base model with different rankers to enable personalized adaptation for diverse user needs simultaneously. The base model runs on high-resource central nodes, while rankers can be deployed on edge devices or even local user devices. The CPU-trainability allows each user’s ranker to perform continual learning with behavioral data, paving the way for deeper personalization.
• Figure : (a) Mathematics Task
• Figure : (a) Mathematics Task