Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Efficiency vs. Accuracy Trade-off: Optimizing RAG-Enhanced LLM Recommender Systems Using Multi-Head Early Exit

Huixue Zhou, Hengrui Gu, Xi Liu, Kaixiong Zhou, Mingfu Liang, Yongkang Xiao, Srinivas Govindan, Piyush Chawla, Jiyan Yang, Xiangfei Meng, Huayu Li, Buyun Zhang, Liang Luo, Wen-Yen Chen, Yiping Han, Bo Long, Rui Zhang, Tianlong Chen

TL;DR

This work tackles the efficiency-accuracy trade-off in RAG-enhanced LLM recommender systems for CTR prediction by coupling a lightweight GCN-Retriever with a multi-head dynamic exiting mechanism. The GCN-Retriever accelerates retrieval by leveraging a bipartite user-item graph and multi-layer embeddings, while the dynamic exiting framework accelerates inference by allowing intermediate heads to output predictive distributions and trigger early termination based on confidence trends. Key contributions include an efficient GCN-based retrieval, an instruction-tuning–driven multi-head exit strategy with a depth-aware learning-rate schedule, and a probability-based early-exit criterion that preserves accuracy while boosting latency performance. The approach demonstrates faster retrieval, robust AUC performance, and improved online responsiveness on real-world datasets, suggesting strong practical potential for real-time LLM-based recommendations.

Abstract

The deployment of Large Language Models (LLMs) in recommender systems for predicting Click-Through Rates (CTR) necessitates a delicate balance between computational efficiency and predictive accuracy. This paper presents an optimization framework that combines Retrieval-Augmented Generation (RAG) with an innovative multi-head early exit architecture to concurrently enhance both aspects. By integrating Graph Convolutional Networks (GCNs) as efficient retrieval mechanisms, we are able to significantly reduce data retrieval times while maintaining high model performance. The early exit strategy employed allows for dynamic termination of model inference, utilizing real-time predictive confidence assessments across multiple heads. This not only quickens the responsiveness of LLMs but also upholds or improves their accuracy, making it ideal for real-time application scenarios. Our experiments demonstrate how this architecture effectively decreases computation time without sacrificing the accuracy needed for reliable recommendation delivery, establishing a new standard for efficient, real-time LLM deployment in commercial systems.

The Efficiency vs. Accuracy Trade-off: Optimizing RAG-Enhanced LLM Recommender Systems Using Multi-Head Early Exit

TL;DR

This work tackles the efficiency-accuracy trade-off in RAG-enhanced LLM recommender systems for CTR prediction by coupling a lightweight GCN-Retriever with a multi-head dynamic exiting mechanism. The GCN-Retriever accelerates retrieval by leveraging a bipartite user-item graph and multi-layer embeddings, while the dynamic exiting framework accelerates inference by allowing intermediate heads to output predictive distributions and trigger early termination based on confidence trends. Key contributions include an efficient GCN-based retrieval, an instruction-tuning–driven multi-head exit strategy with a depth-aware learning-rate schedule, and a probability-based early-exit criterion that preserves accuracy while boosting latency performance. The approach demonstrates faster retrieval, robust AUC performance, and improved online responsiveness on real-world datasets, suggesting strong practical potential for real-time LLM-based recommendations.

Abstract

The deployment of Large Language Models (LLMs) in recommender systems for predicting Click-Through Rates (CTR) necessitates a delicate balance between computational efficiency and predictive accuracy. This paper presents an optimization framework that combines Retrieval-Augmented Generation (RAG) with an innovative multi-head early exit architecture to concurrently enhance both aspects. By integrating Graph Convolutional Networks (GCNs) as efficient retrieval mechanisms, we are able to significantly reduce data retrieval times while maintaining high model performance. The early exit strategy employed allows for dynamic termination of model inference, utilizing real-time predictive confidence assessments across multiple heads. This not only quickens the responsiveness of LLMs but also upholds or improves their accuracy, making it ideal for real-time application scenarios. Our experiments demonstrate how this architecture effectively decreases computation time without sacrificing the accuracy needed for reliable recommendation delivery, establishing a new standard for efficient, real-time LLM deployment in commercial systems.
Paper Structure (14 sections, 13 equations, 4 figures, 4 tables)

This paper contains 14 sections, 13 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Language Models for Recommendation
  4. Efficient Inference
  5. Problem Formulation
  6. GCN-Retriever for Retrieval Acceleration
  7. Dynamic Predictive Exiting for Inference Acceleration
  8. Preliminaries
  9. Optimization of Multi Heads
  10. Workflow of Dynamic Predictive Exiting
  11. Experiments
  12. Conclusion
  13. Limitation
  14. Appendix

Figures (4)

  • Figure 1: The process of GCN-earlyexit: Section 1 GCN-Retriever: Constructs a GCN to generate user embeddings for identifying similar users; Section 2 Construct the Prompt with the Retrieved information: Forms LLM input by combining interaction data of the query user with similar users' data; Section 3 Multi-head Early Exit: Implements an early exit strategy in the LLM by designating potential exit layers and setting probability-based termination criteria.
  • Figure 2: Speedup Comparison between LLM and GCN by Dataset.
  • Figure 3: AUC and RPS by Threshold.
  • Figure 5: AUC and RPS by retrieval examples.