EASELAN: An Open-Source Framework for Multimodal Biosignal Annotation and Data Management

TL;DR

EASELAN addresses the need for large, richly annotated multimodal biosignal datasets by extending the ELAN annotation framework with a three-component, open-source workflow for pre-processing, human-in-the-loop annotation, and post-processing. It integrates Git-based version control and CI/CD for automated validation, supports high-dimensional biosignals, and enables dissemination via NEEMs and SOMA ontology. The framework is demonstrated through the EASE-TSD use case, which comprises extensive multimodal data from table-setting tasks and demonstrates end-to-end data management, annotation, validation, and dissemination, including open-science sharing. Collectively, EASELAN advances reproducibility, cross-lab collaboration, and practical ML and robotics applications by providing a unified, extensible platform for holistic biosignal annotation and data management.

Abstract

Recent advancements in machine learning and adaptive cognitive systems are driving a growing demand for large and richly annotated multimodal data. A prominent example of this trend are fusion models, which increasingly incorporate multiple biosignals in addition to traditional audiovisual channels. This paper introduces the EASELAN annotation framework to improve annotation workflows designed to address the resulting rising complexity of multimodal and biosignals datasets. It builds on the robust ELAN tool by adding new components tailored to support all stages of the annotation pipeline: From streamlining the preparation of annotation files to setting up additional channels, integrated version control with GitHub, and simplified post-processing. EASELAN delivers a seamless workflow designed to integrate biosignals and facilitate rich annotations to be readily exported for further analyses and machine learning-supported model training. The EASELAN framework is successfully applied to a high-dimensional biosignals collection initiative on human everyday activities (here, table setting) for cognitive robots within the DFG-funded Collaborative Research Center 1320 Everyday Activity Science and Engineering (EASE). In this paper we discuss the opportunities, limitations, and lessons learned when using EASELAN for this initiative. To foster research on biosignal collection, annotation, and processing, the code of EASELAN is publicly available(https://github.com/cognitive-systems-lab/easelan), along with the EASELAN-supported fully annotated Table Setting Database.

Figures (3)

• Figure 1: The EASELAN Architecture: (A) Pre-processing component, where the multimodal data input including high-dimensional biosignals are preprocessed and stored in EAF files. (B) Human-in-the-loop Annotation component, where EAF files are synchronized with GitHub for version control and accessed by users to annotate the data with ELAN. These annotations are validated through continuous integration and deployment (CI/CD) pipelines. (C) Post-processing component, where the EAF files together with the validated results are exported into multiple formats that can be further extended for example, to training and validating machine learning models or to retaining Narrative Enabled Episodic Memories (NEEMS), see section 3.
• Figure 2: Screenshort of the EASELAN Dashboard (description see subsection \ref{['dashboard']}).
• Figure 3: Tier overview in HTML-export for validation and testing. Check-marks or red 'X' indicate if a tier is present in an annotation or not present/contains no labels.