Novel Method for Optimizing Performance in Resource Constrained Distributed Data Streams

Simple item page
aut.relation.journalApplied Intelligenceen_NZ
aut.researcherMirza, Farhaan
dc.contributor.authorBhalla, Ren_NZ
dc.contributor.authorPears, Ren_NZ
dc.contributor.authorNaeem, MAen_NZ
dc.contributor.authorMirza, Fen_NZ
dc.date.accessioned2022-11-01T22:44:01Z
dc.date.available2022-11-01T22:44:01Z
dc.description.abstractThe Big Data Era has presented many opportunities for using data mining techniques to discover knowledge patterns across large and diverse collections of data where the volume of data is growing at an exponential rate. Recent approaches to Distributed Data Mining (DDM) have focused on addressing the heterogeneous nature of data sources. However, such approaches do not prioritize the reduction of data communication costs which could be prohibitive in large scale sensor networks where bandwidth is a limited resource. In fact, higher communication and computational costs are the two most prominent problems that have been encountered in heterogeneous distributed environments. Moreover, an effort to decrease the communications load in the distributed environment has an adverse influence on the classification accuracy. Therefore, the research challenge lies in maintaining a balance between transmission cost, computational cost, and accuracy. This paper proposes an algorithm Performance Optimizer in Distributed Stream Mining (PODSM) based on Bayesian Inference to reduce the communication volume and resource time in a heterogeneous distributed data mining environment while retaining prediction accuracy. The approach used in this work exploits the past data for calculating statistics and these statistics are then utilized for the new data. In other words, it imparts the ability to learn from experiences. As a result, our experimental evaluation reveals that a significant reduction in the communication load and an improvement in classification response time can be achieved across a diverse range of dataset types. Reduction of 34.66% was obtained with regard to communication overhead for one of the datasets with huge savings of nearly 27% in resource time. Importantly, instead of showing a negative effect on accuracy, this dataset observes an increment of 0.44% in accuracy.
dc.identifier.citationApplied Intelligence 52, 12924–12942 (2022). https://doi.org/10.1007/s10489-021-03019-5
dc.identifier.doi10.1007/s10489-021-03019-5en_NZ
dc.identifier.issn0924-669Xen_NZ
dc.identifier.issn1573-7497en_NZ
dc.identifier.urihttps://hdl.handle.net/10292/15584
dc.languageenen_NZ
dc.publisherSpringer Science and Business Media LLCen_NZ
dc.relation.urihttps://link.springer.com/article/10.1007/s10489-021-03019-5
dc.rightsAn author may self-archive an author-created version of his/her article on his/her own website and or in his/her institutional repository. He/she may also deposit this version on his/her funder’s or funder’s designated repository at the funder’s request or as a result of a legal obligation, provided it is not made publicly available until 12 months after official publication. He/ she may not use the publisher's PDF version, which is posted on www.springerlink.com, for the purpose of self-archiving or deposit. Furthermore, the author may only post his/her version provided acknowledgement is given to the original source of publication and a link is inserted to the published article on Springer's website. The link must be accompanied by the following text: "The final publication is available at www.springerlink.com”. (Please also see Publisher’s Version and Citation).
dc.rights.accessrightsOpenAccessen_NZ
dc.subjectBig Data; Bayesian inference; Distributed data stream mining; Heterogeneous distributed data
dc.titleNovel Method for Optimizing Performance in Resource Constrained Distributed Data Streamsen_NZ
dc.typeJournal Article
pubs.elements-id449075
pubs.organisational-data/AUT
pubs.organisational-data/AUT/Faculty of Design & Creative Technologies
pubs.organisational-data/AUT/PBRF
pubs.organisational-data/AUT/PBRF/PBRF Design and Creative Technologies
pubs.organisational-data/AUT/PBRF/PBRF Design and Creative Technologies/PBRF ECMS
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
PODSM_WP[96].pdf
Size:
818.24 KB
Format:
Adobe Portable Document Format
Description:
Journal article
Download
License bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
AUT Grant of Licence for Tuwhera Jun 2021.pdf
Size:
360.95 KB
Format:
Adobe Portable Document Format
Description:
Download
Collections
School of Engineering, Computer and Mathematical Sciences - Te Kura Mātai Pūhanga, Rorohiko, Pāngarau