Effect of Activation Function and Model Optimizer on the Performance of Human Activity Recognition System Using Various Deep Learning Models

TL;DR

This study investigates how activation functions (AFs) and model optimizers (MOs) jointly influence deep learning–based human activity recognition (HAR). Using ConvLSTM and BiLSTM on six medically relevant classes drawn from HMDB51 and UCF101, the authors systematically evaluate three AFs (ReLU, Sigmoid, Tanh) against four MOs (SGD, Adam, RMSProp, Adagrad). Key findings show ConvLSTM consistently outperforms BiLSTM, with ConvLSTM plus Adam or RMSProp and ReLU achieving up to about 99% accuracy across datasets, while BiLSTM is dataset-dependent and struggles on HMDB51. The work provides practical AF–MO configuration guidelines for reliable, real-time HAR in healthcare contexts and suggests directions for adaptive optimization and attention-based enhancements in future research.

Abstract

Human Activity Recognition (HAR) plays a vital role in healthcare, surveillance, and innovative environments, where reliable action recognition supports timely decision-making and automation. Although deep learning-based HAR systems are widely adopted, the impact of Activation Functions (AFs) and Model Optimizers (MOs) on performance has not been sufficiently analyzed, particularly regarding how their combinations influence model behavior in practical scenarios. Most existing studies focus on architecture design, while the interaction between AF and MO choices remains relatively unexplored. In this work, we investigate the effect of three commonly used activation functions (ReLU, Sigmoid, and Tanh) combined with four optimization algorithms (SGD, Adam, RMSprop, and Adagrad) using two recurrent deep learning architectures, namely BiLSTM and ConvLSTM. Experiments are conducted on six medically relevant activity classes selected from the HMDB51 and UCF101 datasets, considering their suitability for healthcare-oriented HAR applications. Our experimental results show that ConvLSTM consistently outperforms BiLSTM across both datasets. ConvLSTM, combined with Adam or RMSprop, achieves an accuracy of up to 99.00%, demonstrating strong spatio-temporal learning capabilities and stable performance. While BiLSTM performs reasonably well on UCF101, with accuracy approaching 98.00%, its performance drops to approximately 60.00% on HMDB51, indicating limited robustness across datasets and weaker sensitivity to AF and MO variations. This study provides practical insights for optimizing HAR systems, particularly for real-world healthcare environments where fast and precise activity detection is critical.

Figures (5)

• Figure 1: Percentage distribution of different types of human activities
• Figure 2: Overview of dataset characteristics
• Figure 3: The overview of Proposed Methodology
• Figure 4: The internal architecture of ConvLSTM
• Figure 5: The internal architecture of BiLSTM