Social-RAG: Retrieving from Group Interactions to Socially Ground AI Generation

TL;DR

Social-RAG introduces a retrieval-based grounding framework that uses a social knowledge base built from group interactions to contextualize and tailor AI-generated messages. Implemented as PaperPing in Slack, the approach retrieves social signals, explains recommendations with group-aware context, and learns from reactions to refine future outputs. Field deployment across 18 channels with 500+ researchers shows contextually relevant recommendations that fit existing social practices and foster group common ground, while outlining challenges in balancing group and individual preferences and potential disruptions. The work advances grounded AI by leveraging implicit social cues rather than explicit preferences, offering a scalable path toward socially aware multi-user generation.

Abstract

AI agents are increasingly tasked with making proactive suggestions in online spaces where groups collaborate, yet risk being unhelpful or even annoying if they fail to match group preferences or behave in socially inappropriate ways. Fortunately, group spaces have a rich history of prior interactions and affordances for social feedback that can support grounding an agent's generations to a group's interests and norms. We present Social-RAG, a workflow for socially grounding agents that retrieves context from prior group interactions, selects relevant social signals, and feeds them into a language model to generate messages in a socially aligned manner. We implement this in \textsc{PaperPing}, a system for posting paper recommendations in group chat, leveraging social signals determined from formative studies with 39 researchers. From a three-month deployment in 18 channels reaching 500+ researchers, we observed PaperPing posted relevant messages in groups without disrupting their existing social practices, fostering group common ground.

Figures (7)

• Figure 1: A demonstration of how users interact with PaperPing. Channel members share paper links, react to papers shared by other members or PaperPing and comment on shared papers. After gathering and processing this information, PaperPing sends new contextually grounded paper recommendations, with explanations of how the recommended papers are relevant to the channel. It also provides links to previous related discussions and includes meta-information about the recommended papers in the recommendation message.
• Figure 2: The PaperPing implementation, leveraging the Social RAG workflow.PaperPing extracts relevant social information from two data sources: the Semantic Scholar knowledge base, providing paper content, paper-author connections, and citation graphs; and Slack chat history, which includes previously mentioned papers, emoji reactions, comments, and user interactions (Step 1). PaperPing then retrieves relevant social signals from the data using three heuristics (relevant metadata, the most similar and discussed prior paper in the chat, and the member most likely to be interested) developed over a three-month iterative design process (Step 2). Next, PaperPing uses chains of LLM prompts to turn the retrieved social signals into a natural language explanation (Step 3). Finally, PaperPing posts the message to research group channels, where group members can provide emoji reactions or reply to the message to give feedback (Step 4).
• Figure 3: The PaperPing prompts pipeline (Condition 4). The pipeline starts with social signals leading to three prompts: Prompt 1 highlights paper content; Prompt 2 highlights relevance to a previous paper; and Prompt 3 highlights relevance to a channel member. Finally, outputs from these prompts feed into Prompt 4, which synthesizes the information and adjusts the style, resulting in the final output.
• Figure 4: Four PaperPing conditions shown in the offline evaluation task and example explanations. Names are anonymized.
• Figure 5: Results from post-study questionnaire. Responses are grouped based on the four design goals. We present the original questions posed by the questionnaire in the appendix.