Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SPECTRA: An Efficient Spectral-Informed Neural Network for Sensor-Based Activity Recognition

Deepika Gurung, Lala Shakti Swarup Ray, Mengxi Liu, Bo Zhou, Paul Lukowicz

Abstract

Real time sensor based applications in pervasive computing require edge deployable models to ensure low latency privacy and efficient interaction. A prime example is sensor based human activity recognition where models must balance accuracy with stringent resource constraints. Yet many deep learning approaches treat temporal sensor signals as black box sequences overlooking spectral temporal structure while demanding excessive computation. We present SPECTRA a deployment first co designed spectral temporal architecture that integrates short time Fourier transform STFT feature extraction depthwise separable convolutions and channel wise self attention to capture spectral temporal dependencies under real edge runtime and memory constraints. A compact bidirectional GRU with attention pooling summarizes within window dynamics at low cost reducing downstream model burden while preserving accuracy. Across five public HAR datasets SPECTRA matches or approaches larger CNN LSTM and Transformer baselines while substantially reducing parameters latency and energy. Deployments on a Google Pixel 9 smartphone and an STM32L4 microcontroller further demonstrate end to end deployable realtime private and efficient HAR.

SPECTRA: An Efficient Spectral-Informed Neural Network for Sensor-Based Activity Recognition

Abstract

Real time sensor based applications in pervasive computing require edge deployable models to ensure low latency privacy and efficient interaction. A prime example is sensor based human activity recognition where models must balance accuracy with stringent resource constraints. Yet many deep learning approaches treat temporal sensor signals as black box sequences overlooking spectral temporal structure while demanding excessive computation. We present SPECTRA a deployment first co designed spectral temporal architecture that integrates short time Fourier transform STFT feature extraction depthwise separable convolutions and channel wise self attention to capture spectral temporal dependencies under real edge runtime and memory constraints. A compact bidirectional GRU with attention pooling summarizes within window dynamics at low cost reducing downstream model burden while preserving accuracy. Across five public HAR datasets SPECTRA matches or approaches larger CNN LSTM and Transformer baselines while substantially reducing parameters latency and energy. Deployments on a Google Pixel 9 smartphone and an STM32L4 microcontroller further demonstrate end to end deployable realtime private and efficient HAR.

Paper Structure

This paper contains 22 sections, 11 equations, 1 figure, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. From Classical Approaches to Deep Learning for HAR
  4. Frequency and Spectral Approaches in Deep Learning
  5. Efficient and Edge-oriented HAR
  6. Model Architecture
  7. STFT Feature Extraction
  8. Convolutional Feature Block
  9. Channel Self-Attention
  10. Temporal Modeling with Bi-GRU and Attention Pooling
  11. Classifier and Training Objective
  12. Deployment Optimization
  13. Smartphone Deployment: Google Pixel 9
  14. Microcontroller Deployment: STM32L4S5VIT6
  15. Experimental Evaluation
  16. ...and 7 more sections

Figures (1)

  • Figure 1: SPECTRA architecture: signal-informed spectral--temporal modeling achieves competitive HAR accuracy with a compact, shallow pipeline. Legend: input window $X\in \mathrm{R}^{T\times C}$ ($T$ samples/window, $C$ IMU channels/axes, dataset-dependent); STFT magnitudes $M\in \mathrm{R}^{L\times F\times C}$ ($L$ STFT frames, $F = n_{\mathrm{fft}}/2 + 1$ freq bins); separable conv maps to $L\times F\times D$ features ($D$ conv feature dim); channel self-attention operates over $C$ channels per frame; Bi-GRU with hidden size $H$ and attention pooling aggregates over $L$ to a $2H$ vector; classifier outputs $\hat{\mathbf{y}}$ over $K$ activities.