Large Language Models for Social Networks: Applications, Challenges, and Solutions

TL;DR

This paper presents a production-focused framework for deploying Large Language Models on online social networks by dividing applications into knowledge, engagement, and foundation tasks. It introduces LocalGPT, a neighborhood-specific LLM grounded with real-time social data via a Retrieval-Augmented Generation (RAG) baseline and a novel multi-objective knowledge injection training regime that combines instruction tuning, MCQA, and IRAG to reduce hallucinations and improve grounding, all evaluated with an automated, LM-based evaluation system. For engagement, it proposes a generator-evaluator framework for content like email subject lines, push notifications, and invitation emails, showing that retrieval-grounded, reward-guided generation can improve engagement while highlighting trade-offs in latency, safety, and personalization. Foundation tasks include constructing local knowledge graphs through a teacher-student annotation pipeline and a centralized API to manage LLM usage, enabling scalable, safe, and cost-effective deployment. Collectively, the work demonstrates end-to-end productionizable strategies for leveraging LLMs in hyper-local social networks, with practical implications for up-to-date knowledge grounding, user engagement, and scalable data labeling.

Abstract

Large Language Models (LLMs) are transforming the way people generate, explore, and engage with content. We study how we can develop LLM applications for online social networks. Despite LLMs' successes in other domains, it is challenging to develop LLM-based products for social networks for numerous reasons, and it has been relatively under-reported in the research community. We categorize LLM applications for social networks into three categories. First is knowledge tasks where users want to find new knowledge and information, such as search and question-answering. Second is entertainment tasks where users want to consume interesting content, such as getting entertaining notification content. Third is foundational tasks that need to be done to moderate and operate the social networks, such as content annotation and LLM monitoring. For each task, we share the challenges we found, solutions we developed, and lessons we learned. To the best of our knowledge, this is the first comprehensive paper about developing LLM applications for social networks.

Figures (10)

• Figure 1: An Illustration of the LocalGPT System.
• Figure 2: System Diagram of Index Building.
• Figure 3: The Architecture Design of the RAG-based LocalGPT System.
• Figure 4: The Framework of Knowledge Injection Training. The framework consists of two parts: knowledge injection and evaluation. The knowledge injection pipeline involves two phases: i) Phase I: high-quality data collection through posts and comments ii) Phase II: knowledge injection training through preference learning and instruction-tuning using Q&A variants, MCQA, and IRAG. The Automatic evaluation system contains a question generator for generating both in-sample and out-of-sample questions. The LM-based evaluators evaluate the performance of different models while the guardrail metrics monitor the negative impacts introduced by LMs.
• Figure 5: Question Template Used for Knowledge Injection Training through MCQA Task