ROSAnnotator: A Web Application for ROSBag Data Analysis in Human-Robot Interaction

TL;DR

The paper addresses the challenge of integrating qualitative coding with ROSBag multimodal data in human-robot interaction by introducing ROSAnnotator, a web-based tool that combines manual annotation with multimodal LLM-assisted automation. It details a full workflow including data import, visualization, time-aligned annotation tiers, automatic and manual coding, privacy-preserving preprocessing, and exporting of annotations and statistics. The system is built with a React frontend and Django backend, using a ROSBag processing pipeline to extract video, audio, and transcription data, with open interfaces for custom ROS messages. By providing an open-source, extensible framework for codebooks, multimodal annotation, and dataset construction, ROSAnnotator aims to improve efficiency, reproducibility, and adaptability in qualitative HRI research.

Abstract

Human-robot interaction (HRI) is an interdisciplinary field that utilises both quantitative and qualitative methods. While ROSBags, a file format within the Robot Operating System (ROS), offer an efficient means of collecting temporally synched multimodal data in empirical studies with real robots, there is a lack of tools specifically designed to integrate qualitative coding and analysis functions with ROSBags. To address this gap, we developed ROSAnnotator, a web-based application that incorporates a multimodal Large Language Model (LLM) to support both manual and automated annotation of ROSBag data. ROSAnnotator currently facilitates video, audio, and transcription annotations and provides an open interface for custom ROS messages and tools. By using ROSAnnotator, researchers can streamline the qualitative analysis process, create a more cohesive analysis pipeline, and quickly access statistical summaries of annotations, thereby enhancing the overall efficiency of HRI data analysis. https://github.com/CHRI-Lab/ROSAnnotator

Figures (3)

• Figure 1: This figure illustrates the workflow of ROSAnnotation. (a) The process begins with users collecting ROSBags during an empirical study. (b) Once the ROSBags are imported into ROSAnnotator, multimodal data is extracted. (c) ROSAnnotator supports both automatic and manual annotation. For automatic annotation, users can provide instructions via a chatbox, and ROSAnnotator will perform the annotations accordingly. Users can then make manual adjustments if needed. For manual annotation, users can refer to the codebook, create multiple time axes, generate annotations, and modify the time intervals for annotations. (d) Finally, users can view a statistical summary of their qualitative analysis and extract the resulting annotations.
• Figure 2: The interface of ROSAnnotator WebApp. (a) shows the import page, (b) shows the annotation page with the codebook function in the toolbar, (c) shows other functions in the toolbar
• Figure 3: Code structure of the ROSAnnotator WebApp