Punctual Activity Classification Within Constrained Sporting Domains Using Reservoir Computing

Hunt, Douglas
Parry, David
Narayanan, Ajit
Item type
Degree name
Doctor of Philosophy
Journal Title
Journal ISSN
Volume Title
Auckland University of Technology

This research uses Design Science to create a software artefact that employs recurrent neural networks in the form of Reservoir Computing models such as Liquid State Machines and Echo State Networks to classify rare punctual activity in a sports context. This research represents part of a broader plan to use wearable inertial and other sensors to assist in classifying and coaching human movement.

The research is conducted in three main stages with progress reported via five published papers. The initial stage demonstrates that one form of Reservoir Computing, Liquid State Machines, are capable of classifying "classes" based on synthetic spatio-temporal data. The second stage demonstrates that both Liquid State Machines and Echo State Networks are capable of classifying selected, realistic, punctual human activity normally encountered within equestrian sport. This second stage uses data captured from a wrist mounted inertial sensor used with realistic but scripted activities in laboratory conditions. The third stage utilises data captured from twenty equestrian sports-people undertaking unscripted riding activities in the real world and demonstrates that rare, punctual activity can be successfully classified using an Echo State Network. The real-world data used in the third stage is also captured from a wrist mounted inertial sensor. The punctual activity classified during the third stage of this research represents less than 0.005% of the data captured and so can be said to be "rare".

The main contribution of this research is to demonstrate that it is possible to build a reliable classifier based on spatio-temporal data from punctual human activity using recurrent neural networks in the form of Reservoir Computing models. Reservoir Computing models have been successfully used as classifiers in other areas including speech recognition but have not previously been used to classify human activity. This research concludes that Reservoir Computing models represent a useful adjunct to human activity classifiers but this research does not set out to "prove" that they are necessarily the best or only way of classifying punctual human activities.

A secondary contribution of this research is to extend the differentiation of punctual human activities from durative activities with a cyclic component such as running or rowing and to argue that most prior human activity classification research has focussed on durative activities with much less research focus on short, non-cyclic, punctual activities.

While the classifier artefact developed within this research is intended for use within a sporting context it has other uses beyond this context.

Punctual activity classification , Echo State Networks , Reservoir Computing , Human Activity Classification , Human Activity Recognition , Machine Learning
Publisher's version
Rights statement