Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Scaling Multimodal Search and Recommendation with Small Language Models via Upside-Down Reinforcement Learning

Yu-Chen Lin, Sanat Sharma, Hari Manikandan, Jayant Kumar, Tracy Holloway King, Jing Zheng

TL;DR

This work addresses real-time multimodal search and recommendation by training a compact 104M SLM through synthetic dataset distillation from a larger LLM and optimizing with upside-down reinforcement learning for multitask prompt generation. The approach achieves relevance and diversity close to 8B-LM baselines while delivering dramatic gains in latency and memory efficiency, enabling deployment on standard hardware. It demonstrates competitive performance against Llama-3 8B, Qwen-3 8B, and Ministral 8B at a fraction of parameters, and integrates with real-world image-generation pipelines. These findings highlight the practicality of lightweight, multitask generative engines for scalable multimodal discovery and content creation.

Abstract

In this work, we investigate how small language models (SLMs) can be scaled to support multimodal search and recommendation use cases while remaining efficient enough for real-time, resource-constrained deployments. We present a framework that combines upside-down reinforcement learning with synthetic data distillation from a large language model (Llama-3) to train a 100M-parameter GPT-2 model for multitask prompt generation. Despite being up to 80 times smaller than state-of-the-art large language models (LLMs), our SLM achieves relevance and diversity scores within 6% of competitive baselines such as Llama-3 8B, Qwen3 8B, and Ministral 8B. These results demonstrate that SLMs can effectively handle multimodal search and recommendation tasks, while dramatically reducing inference latency and memory overhead. Our study highlights the potential of lightweight models as practical engines for scalable multimodal discovery, bridging the gap between cutting-edge research and real-world multimodal applications such as media recommendations and creative content generation.

Scaling Multimodal Search and Recommendation with Small Language Models via Upside-Down Reinforcement Learning

TL;DR

This work addresses real-time multimodal search and recommendation by training a compact 104M SLM through synthetic dataset distillation from a larger LLM and optimizing with upside-down reinforcement learning for multitask prompt generation. The approach achieves relevance and diversity close to 8B-LM baselines while delivering dramatic gains in latency and memory efficiency, enabling deployment on standard hardware. It demonstrates competitive performance against Llama-3 8B, Qwen-3 8B, and Ministral 8B at a fraction of parameters, and integrates with real-world image-generation pipelines. These findings highlight the practicality of lightweight, multitask generative engines for scalable multimodal discovery and content creation.

Abstract

In this work, we investigate how small language models (SLMs) can be scaled to support multimodal search and recommendation use cases while remaining efficient enough for real-time, resource-constrained deployments. We present a framework that combines upside-down reinforcement learning with synthetic data distillation from a large language model (Llama-3) to train a 100M-parameter GPT-2 model for multitask prompt generation. Despite being up to 80 times smaller than state-of-the-art large language models (LLMs), our SLM achieves relevance and diversity scores within 6% of competitive baselines such as Llama-3 8B, Qwen3 8B, and Ministral 8B. These results demonstrate that SLMs can effectively handle multimodal search and recommendation tasks, while dramatically reducing inference latency and memory overhead. Our study highlights the potential of lightweight models as practical engines for scalable multimodal discovery, bridging the gap between cutting-edge research and real-world multimodal applications such as media recommendations and creative content generation.

Paper Structure

This paper contains 17 sections, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Synthetic Dataset Distillation
  4. Dataset Curation Process
  5. Upside-Down Reinforcement Learning for SLM Optimization
  6. Reward-Based Prompt Generation
  7. Model Architecture and Key Capabilities
  8. Model Specifications
  9. Training with UDRL
  10. Experiments and Evaluations
  11. Quantitative Evaluation
  12. Relevance Evaluation
  13. Task Adherence Evaluation (Length)
  14. Qualitative Evaluation
  15. Prompt Quality
  16. ...and 2 more sections

Figures (2)

  • Figure 1: The top workstream illustrates our multimodal system with intent detection and prompt generation for contextual search, recommendation, and content generation. The bottom workstream shows our pipeline for training a small language model (SLM) via synthetic data distillation and upside-down reinforcement learning, enabling an efficient real-time alternative to large language models.
  • Figure 2: A sample pipeline for multimodal contextual recommendation incorporating our SLM model for prompt generation.