Models Got Talent: Identifying High Performing Wearable Human Activity Recognition Models Without Training

TL;DR

The paper tackles the challenge of architecture search for sensor-based HAR by introducing Zero Cost Proxies (ZCPs) that predict final network performance from a single forward/backward pass on a random batch, avoiding costly NAS. By evaluating eight ZCPs across six HAR datasets and randomly sampling 1500 CNN/RNN architectures, the study shows that top ZCP-predicted models achieve within approximately $\Delta_1 \approx 0.05$ of the best fully trained model, and training the top-10 predicted models brings this gap down to $\Delta_{10} \lesssim 0.02$, representing large computational savings. The authors also demonstrate that ZCPs maintain competitive ranking (Spearman correlations around 0.5 on several datasets) and exhibit resilience to data noise, with ensemble ZCPs offering robust performance across conditions. These findings indicate that ZCPs can significantly accelerate HAR model development and enable practical deployment of efficient, high-performing architectures in wearable scenarios.

Abstract

A promising alternative to the computationally expensive Neural Architecture Search (NAS) involves the development of Zero Cost Proxies (ZCPs), which correlate well with trained performance, but can be computed through a single forward/backward pass on a randomly sampled batch of data. In this paper, we investigate the effectiveness of ZCPs for HAR on six benchmark datasets, and demonstrate that they discover network architectures that obtain within 5% of performance attained by full-scale training involving 1500 randomly sampled architectures. This results in substantial computational savings as high-performing architectures can be discovered with minimal training. Our experiments not only introduce ZCPs to sensor-based HAR, but also demonstrate that they are robust to data noise, further showcasing their suitability for practical scenarios.

Figures (7)

• Figure 1: The experimental pipeline contains four steps: random architecture generation, computation of the ZCPs, full training of the sampled architectures, and evaluation of the ZCPs.
• Figure 2: Visualizing the $\Delta_1$ and $\Delta_{10}$: for most datasets, the difference in performance between the top predicted model and best trained model is less than 5%. However, training the top 10 predicted models reduces the difference to 1% for some ZCPs.
• Figure 3: Visualizing the talent rate and $\Delta$ for top $10\%$ of predicted models: for most datasets, difference in performance between the best predicted model and those recognized after full scale training are less than 2%.
• Figure 4: Spearman rank correlation between the models ranked by ZCP and full training.
• Figure 5: Impact of data noise on ZCP: increasing the variance of the Gaussian noise has limited impact on the correlation.