NeoMycelia: A Software Reference Architecture for Big Data Systems

Ataei, P
Litchfield, A
Item type
Conference Contribution
Degree name
Journal Title
Journal ISSN
Volume Title

The big data revolution began when the volume, velocity, and variety of data completely overwhelmed the systems used to store, manipulate and analyze that data. As a result, a new class of software systems emerged called big data systems. While many attempted to harness the power of these new systems, it is estimated that approximately 75% of the big data projects have failed within the last decade. One of the root causes of this is software engineering and architecture aspect of these systems. This paper aims to facilitate big data system development by introducing a software reference architecture. The work provides an event driven microservices architecture that addresses specific limitations in current big data reference architectures (RA). The artefact development has followed the principles of empirically grounded RAs. The RA has been evaluated by developing a prototype that solves a real-world problem in practice. At the end, succesful implementation of the reference architecture have been presented. The results displayed a good degree of applicability with respect to Quality factors.

Reference architecture; Architecture; Big data reference architecture; Big data architecture; Big data systems; Big data software engineering; Event driven; Microservices
In 2021 28th Asia-Pacific Software Engineering Conference (APSEC), Taipei, Taiwan, 6-9 Dec. 2021, doi:
Rights statement
Copyright © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.