Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sensor-Specific Transformer (PatchTST) Ensembles with Test-Matched Augmentation

Pavankumar Chandankar, Robin Burchard

TL;DR

This work tackles robust human activity recognition from noisy, multi-sensor wearables by constructing sensor-specific PatchTST encoders and training them with test-matched augmentations to emulate real-world perturbations. A dual-stream, probability-level ensemble across four limbs, augmented with calibration and class-imbalance handling, yields strong macro-F1 gains on the WEAR challenge, including a private leaderboard score of 0.5172. The approach emphasizes lightweight transformer-based encoders suitable for edge deployment and demonstrates resilience to single-sensor drop-out while maintaining high predictive accuracy. Overall, the study shows that aligning training-time noise with test-time conditions and fusing per-sensor predictions can substantially improve HAR robustness in wild settings.

Abstract

We present a noise-aware, sensor-specific ensemble approach for robust human activity recognition on the 2nd WEAR Dataset Challenge. Our method leverages the PatchTST transformer architecture, training four independent models-one per inertial sensor location-on a tampered training set whose 1-second sliding windows are augmented to mimic the test-time noise. By aligning the train and test data schemas (JSON-encoded 50-sample windows) and applying randomized jitter, scaling, rotation, and channel dropout, each PatchTST model learns to generalize across real-world sensor perturbations. At inference, we compute softmax probabilities from all four sensor models on the Kaggle test set and average them to produce final labels. On the private leaderboard, this pipeline achieves a macro-F1 substantially above the baseline, demonstrating that test-matched augmentation combined with transformer-based ensembling is an effective strategy for robust HAR under noisy conditions.

Sensor-Specific Transformer (PatchTST) Ensembles with Test-Matched Augmentation

TL;DR

This work tackles robust human activity recognition from noisy, multi-sensor wearables by constructing sensor-specific PatchTST encoders and training them with test-matched augmentations to emulate real-world perturbations. A dual-stream, probability-level ensemble across four limbs, augmented with calibration and class-imbalance handling, yields strong macro-F1 gains on the WEAR challenge, including a private leaderboard score of 0.5172. The approach emphasizes lightweight transformer-based encoders suitable for edge deployment and demonstrates resilience to single-sensor drop-out while maintaining high predictive accuracy. Overall, the study shows that aligning training-time noise with test-time conditions and fusing per-sensor predictions can substantially improve HAR robustness in wild settings.

Abstract

We present a noise-aware, sensor-specific ensemble approach for robust human activity recognition on the 2nd WEAR Dataset Challenge. Our method leverages the PatchTST transformer architecture, training four independent models-one per inertial sensor location-on a tampered training set whose 1-second sliding windows are augmented to mimic the test-time noise. By aligning the train and test data schemas (JSON-encoded 50-sample windows) and applying randomized jitter, scaling, rotation, and channel dropout, each PatchTST model learns to generalize across real-world sensor perturbations. At inference, we compute softmax probabilities from all four sensor models on the Kaggle test set and average them to produce final labels. On the private leaderboard, this pipeline achieves a macro-F1 substantially above the baseline, demonstrating that test-matched augmentation combined with transformer-based ensembling is an effective strategy for robust HAR under noisy conditions.
Paper Structure (20 sections, 13 equations, 4 figures, 6 tables)

This paper contains 20 sections, 13 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. 2nd WEAR Dataset Challenge
  3. Sensor configuration and recording protocol
  4. Train/test split
  5. Task formulation and evaluation metric
  6. Model Architecture and Training
  7. Window extraction and normalisation
  8. Noise-Aware Data Augmentation
  9. Sensor-Specific PatchTST Encoder
  10. Training and Probability-Level Ensembling
  11. Experimental Results
  12. Experimental Protocol
  13. Leaderboard Comparison
  14. Ablation Study
  15. Robustness to Sensor Drop-out
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Segmentation with 50-sample windows. Top: 50 % overlap used in training; bottom: non-overlapping windows used at inference.
  • Figure 2: Effect of the four stochastic transforms used during training on a single 1s accelerometer window (x–axis shown): (a) Raw signal; (b) Gaussian jitter; (c)Amplitude scaling; (d) Small-angle rotation; (e) Channel dropout. The transforms preserve overall waveform shape while injecting realistic noise patterns.
  • Figure 3: PatchTST architecture. (a) Sensor-specific PatchTST encoder: an input $50\times3$ accelerometer window is instance-normalised, split into patches, linearly projected with positional embeddings, processed by $N$ stacked Transformer layers, flattened, and mapped to 19 activity logits. (b) Internal architecture of one pre-norm Transformer layer, consisting of multi-head self-attention, feed-forward network, and residual Add + Norm blocks.
  • Figure 4: Normalized confusion matrix of the dual-stream ensemble on the hidden test set.