Frequency-aware Spatio-temporal Topology Learning for Skeleton-based Human Activity Recognition
Date
Authors
Xia, Y
Yongchareon, S
Lutui, R
Sheng, QZ
Supervisor
Item type
Journal Article
Degree name
Journal Title
Journal ISSN
Volume Title
Publisher
Elsevier BV
Abstract
Skeleton-based human activity recognition (HAR) has made significant progress through graph convolutional networks (GCNs) and Transformer architectures for spatiotemporal modeling. However, existing methods either employ predefined static graph topologies that cannot adapt to heterogeneous skeleton data or learn dynamic topologies based solely on local spatiotemporal features, thereby overlooking the global temporal frequency features of joint movements that are important for discovering semantically meaningful spatial relationships. We propose Frequency-Aware Topology Learning Graph Convolutional Network (FATL-GCN), a novel architecture that integrates frequency-aware temporal context to guide adaptive learning of spatial topology. Our approach leverages Time-to-Vector linear frequency encoding to capture both periodic and non-periodic motion patterns, employs frequency-guided topology learning to generate action-specific graphs through temporal-context-driven attention, and incorporates hierarchical multi-scale fusion for robust feature extraction across scales. Extensive experiments achieved top-1 accuracies of 93.8% (cross-subject) and 97.5% (cross-view) on NTU-60, 91.9% (cross-subject) and 93.1% (cross-setup) on NTU-120, and 51.7% on Kinetics-Skeleton. Ablation studies confirm the critical role of our components, with removing the dynamic graph topology causing a 3.5% accuracy drop and removing frequency-aware encoding causing a 2.1% drop.Description
Keywords
46 Information and Computing Sciences, 4607 Graphics, Augmented Reality and Games, Bioengineering, Networking and Information Technology R&D (NITRD), 0801 Artificial Intelligence and Image Processing, 0806 Information Systems, 0906 Electrical and Electronic Engineering, Artificial Intelligence & Image Processing, 4603 Computer vision and multimedia computation, 4605 Data management and data science, 4611 Machine learning
Source
Pattern Recognition, ISSN: 0031-3203 (Print), Elsevier BV, 175, 113146-113146. doi: 10.1016/j.patcog.2026.113146
Publisher's version
Rights statement
© 2026 The Author(s). Published by Elsevier Ltd. Creative Commons. This is an open access article distributed under the terms of the Creative Commons CC-BY license, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. You are not required to obtain permission to reuse this article.