From SERPs to Agents: A Platform for Comparative Studies of Information Interaction

TL;DR

UXLab addresses the engineering barrier of conducting comparative studies across traditional search, retrieval-augmented generation, and agentic information systems by introducing an open-source, no-code dashboard. The four-part architecture—Backend, Experimenter Dashboard, Participant Interface, and Service Connectors—enables end-to-end study design, deployment, and high-fidelity logging. A within-subject micro-study comparing RAG and Agentic search demonstrates the platform's ability to run complex experiments in hours with reproducible configurations. This work lowers the barrier to systematic human-information interaction research and supports future multi-modal investigations.

Abstract

The diversification of information access systems, from RAG to autonomous agents, creates a critical need for comparative user studies. However, the technical overhead to deploy and manage these distinct systems is a major barrier. We present UXLab, an open-source system for web-based user studies that addresses this challenge. Its core is a web-based dashboard enabling the complete, no-code configuration of complex experimental designs. Researchers can visually manage the full study, from recruitment to comparing backends like traditional search, vector databases, and LLMs. We demonstrate UXLab's value via a micro case study comparing user behavior with RAG versus an autonomous agent. UXLab allows researchers to focus on experimental design and analysis, supporting future multi-modal interaction research.

Figures (2)

• Figure 1: Overview of the UXLab Experimenter Dashboard.
• Figure 2: UXLab system architecture. The Experimenter Dashboard configures the Backend, which routes all requests from the Participant Interface via Service Connectors to the appropriate external or local services.