Parameter-Efficient Fine-Tuning for HAR: Integrating LoRA and QLoRA into Transformer Models

TL;DR

This work tackles the challenge of adapting large transformer-based HAR models under resource constraints by evaluating parameter-efficient fine-tuning methods LoRA and QLoRA on a MAE-pretrained backbone. Using a Leave-One-Dataset-Out protocol across five HAR datasets, the study shows that LoRA and QLoRA achieve near full fine-tuning accuracy while drastically reducing trainable parameters and memory usage, with adapter rank and quantization offering controllable trade-offs. The findings demonstrate practical pathways for edge-ready HAR systems and highlight robustness to limited supervision, particularly for LoRA. Potential future directions include extending to federated, online continual learning, and automated adapter/quantization configuration to suit deployment constraints.

Abstract

Human Activity Recognition is a foundational task in pervasive computing. While recent advances in self-supervised learning and transformer-based architectures have significantly improved HAR performance, adapting large pretrained models to new domains remains a practical challenge due to limited computational resources on target devices. This papers investigates parameter-efficient fine-tuning techniques, specifically Low-Rank Adaptation (LoRA) and Quantized LoRA, as scalable alternatives to full model fine-tuning for HAR. We propose an adaptation framework built upon a Masked Autoencoder backbone and evaluate its performance under a Leave-One-Dataset-Out validation protocol across five open HAR datasets. Our experiments demonstrate that both LoRA and QLoRA can match the recognition performance of full fine-tuning while significantly reducing the number of trainable parameters, memory usage, and training time. Further analyses reveal that LoRA maintains robust performance even under limited supervision and that the adapter rank provides a controllable trade-off between accuracy and efficiency. QLoRA extends these benefits by reducing the memory footprint of frozen weights through quantization, with minimal impact on classification quality.

Figures (2)

• Figure 1: Detailed Encoder Architecture.
• Figure 2: F1 Macro and training time as a function of LoRA adapter rank.