Retrieval-Augmented Simulacra: Generative Agents for Up-to-date and Knowledge-Adaptive Simulations
Hikaru Shimadzu, Takehito Utsuro, Daisuke Kitayama
TL;DR
This paper addresses predicting trends in social networking services (SNS) by building a multi-agent SNS simulator powered by large language models (LLMs). It introduces a retrieval-augmented generation (RAG) mechanism to fetch and summarize up-to-date information for posts and replies, enabling topic-specific, realistic discussions. Through experiments comparing no RAG, simple RAG, and an advanced RAG method across two scenarios, the authors find that the proposed RAG approach yields the most natural thread-level exchanges, though conformity to community goals and rules can vary. The work highlights the importance of information-source selection and points to future improvements in autonomous source selection to broaden scenario coverage and quality of simulations.
Abstract
In the 2023 edition of the White Paper on Information and Communications, it is estimated that the population of social networking services in Japan will exceed 100 million by 2022, and the influence of social networking services in Japan is growing significantly. In addition, marketing using SNS and research on the propagation of emotions and information on SNS are being actively conducted, creating the need for a system for predicting trends in SNS interactions. We have already created a system that simulates the behavior of various communities on SNS by building a virtual SNS environment in which agents post and reply to each other in a chat community created by agents using a LLMs. In this paper, we evaluate the impact of the search extension generation mechanism used to create posts and replies in a virtual SNS environment using a simulation system on the ability to generate posts and replies. As a result of the evaluation, we confirmed that the proposed search extension generation mechanism, which mimics human search behavior, generates the most natural exchange.