Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ALS-HAR: Harnessing Wearable Ambient Light Sensors to Enhance IMU-based Human Activity Recogntion

Lala Shakti Swarup Ray, Daniel Geißler, Mengxi Liu, Bo Zhou, Sungho Suh, Paul Lukowicz

TL;DR

This work investigates wearable ambient light sensors (ALS) as a modality for human activity recognition (HAR) and addresses their sensitivity to ambient lighting by transferring knowledge from ALS to IMU-based HAR. The authors introduce LightHAR, an ALS-only classifier, and two ALS-informed inertial HAR strategies (MultiLight InertialHAR and ContraLight InertialHAR) that enable robust activity recognition using IMU data at inference while leveraging ALS during training. A new multi-modal dataset with wrist ALS, wrist IMU, video, and SMPL pose across three lighting scenarios supports evaluation, showing that cross-modal transfer improves IMU-based HAR by up to 4.2 percentage points in accuracy and 6.4 percentage points in macro F1, and can surpass some multi-modal fusion baselines in certain conditions. The findings demonstrate the untapped potential of ALS to enhance sensor-based HAR, enabling practical, energy-efficient wearables for applications in healthcare, sports monitoring, and smart indoor environments, while highlighting the need for cross-device and multi-ALS validations.

Abstract

Despite the widespread integration of ambient light sensors (ALS) in smart devices commonly used for screen brightness adaptation, their application in human activity recognition (HAR), primarily through body-worn ALS, is largely unexplored. In this work, we developed ALS-HAR, a robust wearable light-based motion activity classifier. Although ALS-HAR achieves comparable accuracy to other modalities, its natural sensitivity to external disturbances, such as changes in ambient light, weather conditions, or indoor lighting, makes it challenging for daily use. To address such drawbacks, we introduce strategies to enhance environment-invariant IMU-based activity classifications through augmented multi-modal and contrastive classifications by transferring the knowledge extracted from the ALS. Our experiments on a real-world activity dataset for three different scenarios demonstrate that while ALS-HAR's accuracy strongly relies on external lighting conditions, cross-modal information can still improve other HAR systems, such as IMU-based classifiers.Even in scenarios where ALS performs insufficiently, the additional knowledge enables improved accuracy and macro F1 score by up to 4.2 % and 6.4 %, respectively, for IMU-based classifiers and even surpasses multi-modal sensor fusion models in two of our three experiment scenarios. Our research highlights the untapped potential of ALS integration in advancing sensor-based HAR technology, paving the way for practical and efficient wearable ALS-based activity recognition systems with potential applications in healthcare, sports monitoring, and smart indoor environments.

ALS-HAR: Harnessing Wearable Ambient Light Sensors to Enhance IMU-based Human Activity Recogntion

TL;DR

This work investigates wearable ambient light sensors (ALS) as a modality for human activity recognition (HAR) and addresses their sensitivity to ambient lighting by transferring knowledge from ALS to IMU-based HAR. The authors introduce LightHAR, an ALS-only classifier, and two ALS-informed inertial HAR strategies (MultiLight InertialHAR and ContraLight InertialHAR) that enable robust activity recognition using IMU data at inference while leveraging ALS during training. A new multi-modal dataset with wrist ALS, wrist IMU, video, and SMPL pose across three lighting scenarios supports evaluation, showing that cross-modal transfer improves IMU-based HAR by up to 4.2 percentage points in accuracy and 6.4 percentage points in macro F1, and can surpass some multi-modal fusion baselines in certain conditions. The findings demonstrate the untapped potential of ALS to enhance sensor-based HAR, enabling practical, energy-efficient wearables for applications in healthcare, sports monitoring, and smart indoor environments, while highlighting the need for cross-device and multi-ALS validations.

Abstract

Despite the widespread integration of ambient light sensors (ALS) in smart devices commonly used for screen brightness adaptation, their application in human activity recognition (HAR), primarily through body-worn ALS, is largely unexplored. In this work, we developed ALS-HAR, a robust wearable light-based motion activity classifier. Although ALS-HAR achieves comparable accuracy to other modalities, its natural sensitivity to external disturbances, such as changes in ambient light, weather conditions, or indoor lighting, makes it challenging for daily use. To address such drawbacks, we introduce strategies to enhance environment-invariant IMU-based activity classifications through augmented multi-modal and contrastive classifications by transferring the knowledge extracted from the ALS. Our experiments on a real-world activity dataset for three different scenarios demonstrate that while ALS-HAR's accuracy strongly relies on external lighting conditions, cross-modal information can still improve other HAR systems, such as IMU-based classifiers.Even in scenarios where ALS performs insufficiently, the additional knowledge enables improved accuracy and macro F1 score by up to 4.2 % and 6.4 %, respectively, for IMU-based classifiers and even surpasses multi-modal sensor fusion models in two of our three experiment scenarios. Our research highlights the untapped potential of ALS integration in advancing sensor-based HAR technology, paving the way for practical and efficient wearable ALS-based activity recognition systems with potential applications in healthcare, sports monitoring, and smart indoor environments.
Paper Structure (18 sections, 3 equations, 4 figures, 3 tables)

This paper contains 18 sections, 3 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. ALS-HAR
  4. Knowledge Transfer in Sensor HAR
  5. Data Collection
  6. Method
  7. LightHAR
  8. Light embedded InertialHAR:
  9. MultiLight InertialHAR
  10. ContraLight InertialHAR
  11. Evaluation
  12. Training details
  13. Unimodal Results
  14. Cross-modal Results
  15. Discussions
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Synchronized ambient light and IMU accelerometer signals extracted from the collected dataset aligned with the some of the labelled activity classes.
  • Figure 2: Overview of the architecture of LightHAR that uses ALS data only for activity classification.
  • Figure 3: Overview of MultiLight InertialHAR that takes both ALS and IMU data during training and relies on IMU only during inference by filling the ALS part with zeros as placeholder during inference.
  • Figure 4: Overview of ContraLight InertialHAR that takes both ALS and IMU data during training but only IMU during inference.