Integrating Spatial and Temporal Information for Violent Activity Detection from Video Using Deep Spiking Neural Networks

Simple item page
aut.relation.endpage4532
aut.relation.issue9
aut.relation.journalSensors (Basel)
aut.relation.startpage4532
aut.relation.volume23
dc.contributor.authorWang, Xiang
dc.contributor.authorYang, Jie
dc.contributor.authorKasabov, Nikola K
dc.date.accessioned2023-07-03T00:50:17Z
dc.date.available2023-07-03T00:50:17Z
dc.date.issued2023-05-06
dc.description.abstractIncreasing violence in workplaces such as hospitals seriously challenges public safety. However, it is time- and labor-consuming to visually monitor masses of video data in real time. Therefore, automatic and timely violent activity detection from videos is vital, especially for small monitoring systems. This paper proposes a two-stream deep learning architecture for video violent activity detection named SpikeConvFlowNet. First, RGB frames and their optical flow data are used as inputs for each stream to extract the spatiotemporal features of videos. After that, the spatiotemporal features from the two streams are concatenated and fed to the classifier for the final decision. Each stream utilizes a supervised neural network consisting of multiple convolutional spiking and pooling layers. Convolutional layers are used to extract high-quality spatial features within frames, and spiking neurons can efficiently extract temporal features across frames by remembering historical information. The spiking neuron-based optical flow can strengthen the capability of extracting critical motion information. This method combines their advantages to enhance the performance and efficiency for recognizing violent actions. The experimental results on public datasets demonstrate that, compared with the latest methods, this approach greatly reduces parameters and achieves higher inference efficiency with limited accuracy loss. It is a potential solution for applications in embedded devices that provide low computing power but require fast processing speeds.
dc.identifier.citationSensors (Basel), ISSN: 1424-8220 (Print); 1424-8220 (Online), MDPI AG, 23(9), 4532-4532. doi: 10.3390/s23094532
dc.identifier.doi10.3390/s23094532
dc.identifier.issn1424-8220
dc.identifier.issn1424-8220
dc.identifier.urihttp://hdl.handle.net/10292/16347
dc.languageeng
dc.publisherMDPI AG
dc.relation.urihttps://www.mdpi.com/1424-8220/23/9/4532
dc.rights.accessrightsOpenAccess
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectdeep learning
dc.subjectoptical flow
dc.subjectspatial and temporal analysis
dc.subjectspiking neural network
dc.subjectviolence detection
dc.subjectdeep learning
dc.subjectoptical flow
dc.subjectspatial and temporal analysis
dc.subjectspiking neural network
dc.subjectviolence detection
dc.subject4611 Machine Learning
dc.subject46 Information and Computing Sciences
dc.subject4605 Data Management and Data Science
dc.subject4603 Computer Vision and Multimedia Computation
dc.subjectBioengineering
dc.subject0301 Analytical Chemistry
dc.subject0502 Environmental Science and Management
dc.subject0602 Ecology
dc.subject0805 Distributed Computing
dc.subject0906 Electrical and Electronic Engineering
dc.subjectAnalytical Chemistry
dc.subject3103 Ecology
dc.subject4008 Electrical engineering
dc.subject4009 Electronics, sensors and digital hardware
dc.subject4104 Environmental management
dc.subject4606 Distributed computing and systems software
dc.subject.meshMotion
dc.subject.meshNeural Networks, Computer
dc.subject.meshNeurons
dc.subject.meshOptic Flow
dc.subject.meshNeural Networks, Computer
dc.subject.meshNeurons
dc.subject.meshOptic Flow
dc.subject.meshMotion
dc.titleIntegrating Spatial and Temporal Information for Violent Activity Detection from Video Using Deep Spiking Neural Networks
dc.typeJournal Article
pubs.elements-id506478
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Wang et al_2023_Integrating spatial and temporal information.pdf
Size:
13.74 MB
Format:
Adobe Portable Document Format
Description:
Journal article
Download
Collections
KEDRI - the Knowledge Engineering and Discovery Research Institute