LUIDA: Large-scale Unified Infrastructure for Digital Assessments based on Commercial Metaverse Platform

TL;DR

LUIDA tackles fragmentation in VR/HCI research workflows by unifying recruitment, experiment execution, and data collection inside metaverse environments. It introduces a platform-agnostic architecture with five components and a Unity-based Implementation Template, validated on Cluster through a VR researcher usability study and three replicated large-scale demonstrations (~200 participants per study). The key contributions include automated recruitment, parallel experiment execution, in-VR questionnaires, continuous data capture, a state-machine workflow, and dynamic avatar management, showing high usability for researchers and maintained experimental validity across perception, Proteus-effect, and 3D Fitts' Law paradigms. This approach enables scalable, remote VR studies with robust data integrity, suggesting substantial practical impact for accelerating VR research while leveraging metaverse ecosystems.

Abstract

Online experiments using metaverse platforms have gained significant traction in Human-Computer Interaction and Virtual Reality (VR) research. However, current research workflows are highly fragmented, as researchers must use separate tools for system implementation, participant recruitment, experiment execution, and data collection, reducing consistency and increasing workload. We present LUIDA (Large-scale Unified Infrastructure for Digital Assessments), a metaverse-based framework that integrates these fragmented processes. LUIDA automatically allocates interconnected virtual environments for parallel experiment execution and provides implementation templates adaptable to various VR research domains, requiring minimal metaverse development expertise. Our evaluation included two studies using a prototype built on Cluster, the commercial metaverse platform. First, VR researchers using LUIDA to develop and run experiments reported high usability scores (SUS: 73.75) and moderate workload (NASA-TLX: 24.11) for overall usage, with interviews confirming streamlined workflows compared to traditional laboratory experiments. Second, we conducted three replicated experiments with public Cluster users, each recruiting approximately 200 participants within one week. These experiments produced results that closely matched the original studies, validating the experimental integrity of LUIDA across research domains. After technical refinements, we plan to release LUIDA as an open platform, providing a standardized protocol to improve research efficiency and experimental reproducibility in VR studies.

Figures (11)

• Figure 1: An overview of the LUIDA system architecture. LUIDA consists of five components: (A) "Web Console", (B) "Storage", (C) "Implementation Template", (D) "Experiment World", and (E) "Recruitment World". (From left) Researchers first design experiments using the Implementation Template, which provides interfaces for variable handling, state machine management, and data format definition. These experiments are registered in the Web Console, where researchers can specify recruitment requirements and questionnaire content. Once completed, Experiment Worlds are uploaded to the Metaverse platform and made available to participants through the Recruitment World. (From right) Participants browse available experiments, select those of interest, and participate through dedicated portals. During the experiment, questionnaires are automatically displayed and data is systematically recorded in the storage component. Researchers can monitor results through the Web Console, which provides visualization and download capabilities.
• Figure 2: Participant experience using LUIDA: (1) Participants enter the Recruitment World; (2) they browse available experiments; (3) they select an experiment and confirm its details; (4) upon pressing the join button, a portal to the Experiment World appears, and participants step onto it to transition; (5) participants read and submit the consent form; (6) participants engage in the experiment (for details of each experiment, see Figs. \ref{['fig:hand-redirection-exp-procedure']}, \ref{['fig:drumming-avatar-exp-procedure']}, and \ref{['fig:fitts-3d-exp-procedure']}); (7) after completing the experiment, participants step onto a world portal to return to the Recruitment World and receive their reward.
• Figure 3: Configuration procedure for automated questionnaire generation: (1) In the LUIDA Web Console, users register a questionnaire using a form. (2) From the list of registered questionnaires, they click the "Questions" button to edit the question items (3). (4) In the Implementation Template (the screenshot captures the actual interface within the Implementation Template of LUIDA's prototype), the corresponding questionnaire ID is set within the state where the questionnaire should appear. (5) During the experiment, the Experiment World automatically generates questionnaire objects based on the registered question items (3).
• Figure 4: Configuration of the centralized finite state machine that manages the entire experiment flow. (A) Example setup illustrating state transitions; arrows without text indicate transitions to be triggered manually (e.g., when participants click a button or complete a questionnaire). (B) Interface for configuring these settings within the Implementation Template of LUIDA’s prototype.
• Figure 5: Configuration of objects listening to state transitions, leveraging the observer pattern. (A) Each in‑scene object can hold multiple state listeners, each monitoring a specific state. When the monitored state is activated, the corresponding listener is triggered and executes its assigned actions. (B) An example of a state listener held by an in‑scene object, defining actions for entering the monitored state, actions to be performed each frame while in the state, and actions for leaving the state. (C) The interface used to configure these settings within the Implementation Template of LUIDA’s prototype, comprising: (C1) a list of in‑scene objects, (C2) a list of state listeners, and (C3) the actions defined within each listener.