Repository logo
 

Sign Language Recognition from Digital Videos Using Pyramid Network with Detection Transformer

Files

Thesis(2.92 MB)

Date

2022

Authors

Supervisor

Nand, Parma
Yan, Weiqi

Item type

Thesis

Degree name

Master of Computer and Information Sciences

Journal Title

Journal ISSN

Volume Title

Publisher

Auckland University of Technology

Abstract

Sign language recognition is one of the fundamental ways to assist deaf people to communicate with others. An accurate visual-based sign language recognition system using deep learning is a long-term research goal. Deep convolutional neural networks have been extensively considered in the last few years, many architectures have been proposed. Recently, Vision Transformer and other Transformers have shown apparent advantages in object recognition compared to traditional Computer Vision models such as Faster R-CNN, YOLO, SSD, and other deep learning models. In this thesis, we propose a Vision Transformer-based sign language recognition method related to DETR (Detection Transformer) to improve the current state-of-the-art sign language recognition accuracy. The method proposed in this thesis is able to recognize sign language from digital videos with high accuracy using a new Deep Learning model, ResNet152 + FPN (i.e., Feature Pyramid Network), which is based on Detection Transformer. Our experiments show that the method has excellent potential for improving sign language recognition accuracy. For instance, our newly proposed net ResNet152+FPN is able to enhance the detection accuracy by up to 1.70% on the test dataset of sign language. Besides, an overall accuracy 96.45% was achieved using the proposed method.

Description

Keywords

Sign language recognition; ResNet152; Detection Transformer; Feature pyramid network

Source

DOI

Publisher's version

Rights statement

Permanent link

https://hdl.handle.net/10292/16092

Collections

Masters Theses