SKS-Transformer: Multi-scale and Direction-aware Attention for Inertial Sensor-based Activity Recognition

Abstract

Introduction: Human Activity Recognition (HAR) has emerged as an enabling research field, with applications ranging from healthcare and sports analytics to smart environments. However, achieving scalable and accurate HAR systems that generalize across diverse activity scenarios remains a challenging problem.

Methods: In this paper, we propose a scalable HAR system, which integrates a new model named SKS-Transformer with a custom-designed wearable Inertial Measurement Unit (IMU). The IMU combines an ESP8266 microcontroller and a JY61 sensor, enabling wireless acquisition of motion data. The proposed SKS-Transformer model incorporates Selective Kernel Networks and squeeze-enhanced axial attention modules to capture multiscale temporal dynamics and directional dependencies, respectively. The motion data preprocessing pipeline includes denoising, segmentation, and normalization. The preprocessed data are fused through a learnable gating mechanism, enabling the model to adaptively balance local and global motion patterns.

Results: We evaluate the system scalability and performance on two public datasets (UCI-HAR and PAMAP2) and two captured datasets that feature both daily activities and fine-grained golf swing errors. Experimental results demonstrate that the SKS-Transformer model consistently surpasses the state of the art on both public datasets (by 0.3% and 0.09% compared to the best of 11 other published models) and by 2.86% and 0.46%, achieving the accuracy of up to 98.10% on collected HAR data, as well as 100% accuracy in golf swing error detection.

Discussion: Ablation studies of SKS-Transformer confirm the contribution of each architectural model component to overall model performance and provide further insights for future optimizations. Future work will investigate the applications of the SKS-Transformer-based system in extended real-world scenarios, including intelligent healthcare, sports performance monitoring, and wearable computing. The source code for our proposed method has been released publicly and is available on GitHub at: URL: https://github.com/cw-feng/SKS-Transformer-Multi-scale-and-direction-aware-attention-for-activity-recognition