Thalamus: A User Simulation Toolkit for Prototyping Multimodal Sensing Studies

TL;DR

Thalamus tackles the high cost and complexity of prototyping multimodal sensing experiments by providing a Python-based toolkit that can capture, simulate, and synchronize diverse data streams across devices. Its three-component architecture (Thalamus Core, Recording devices, and Clients) uses socket-based communication and JSON messaging to enable cross-device prototyping, while built-in features like missing-value handling, filtering, synchronization, noise injection, and delay modeling mimic real-world data conditions. The paper demonstrates practical use cases—real-time applications, cost-aware device selection, stress-testing, remote data collection, and broadcast scenarios—to show the tool’s versatility and impact for HCI research. By offering an open-source solution for dry-runs and robustness testing, Thalamus promises to reduce time, cost, and risk in the design and evaluation of multimodal sensing studies.

Abstract

Conducting user studies that involve physiological and behavioral measurements is very time-consuming and expensive, as it not only involves a careful experiment design, device calibration, etc. but also a careful software testing. We propose Thalamus, a software toolkit for collecting and simulating multimodal signals that can help the experimenters to prepare in advance for unexpected situations before reaching out to the actual study participants and even before having to install or purchase a specific device. Among other features, Thalamus allows the experimenter to modify, synchronize, and broadcast physiological signals (as coming from various data streams) from different devices simultaneously and not necessarily located in the same place. Thalamus is cross-platform, cross-device, and simple to use, making it thus a valuable asset for HCI research.

Figures (6)

• Figure 1: Conceptual diagram of the Thalamus toolkit, illustrating how it can be used to simulate different types of recording devices and synchronize various data streams, including e.g. brain signals and eye movements. Thalamus can receive feedback from any device. For example, here client #1 can also act as a recording device (Recording Device #5), thereby enabling real-time multimodal data collection and analysis.
• Figure 2: Demonstration of the toolkit's ability to handle missing values. This figure illustrates the pupil size values obtained from a sample of eye-tracking data. Missing values represented by "NA" are replaced with zeros.
• Figure 3: Demonstration of the toolkit's ability to provide common filters. An original signal, in this case the mouse cursor position in the X axis, is show on the left part. The same signal after being filtered with the Savitzky-Golay filter is shown on the right.
• Figure 4: Demonstration of the toolkit's ability to synchronize simultaneous data streams. The upper timeseries represents the brain signal (EEG) whereas the lower timeseries represents the pupil diameter.
• Figure 5: Demonstration of the toolkit's ability to simulate noise. An original timeseries, in this case the mouse cursor position in the X axis, is shown on the left part. The same signal is incorporated Gaussian noise, as shown on the right part.