Data provenance and management in Radio Astronomy: a stream computing approach

Simple item page
aut.researcherEnsor, Andrew
dc.contributor.authorMahmoud, M
dc.contributor.authorEnsor, A
dc.contributor.authorBiem, A
dc.contributor.authorElmegreen, B
dc.contributor.authorand Gulyaev, S
dc.contributor.editorTaylor, J
dc.contributor.editorGiugni, S
dc.contributor.editorWilliamson, D
dc.contributor.editorLiu, Q
dc.contributor.editorBai, Q
dc.date.accessioned2011-12-16T03:25:00Z
dc.date.available2011-12-16T03:25:00Z
dc.date.copyright2011
dc.date.issued2011
dc.description.abstractNew approaches for data provenance and data management (DPDM) are required for mega science projects like the Square Kilometer Array, characterized by extremely large data volume and intense data rates, therefore demanding innovative and highly efficient computational paradigms. In this context, we explore a stream-computing approach with the emphasis on the use of accelerators. In particular, we make use of a new generation of high performance stream-based parallelization middleware known as InfoSphere Streams. Its viability for managing and ensuring interoperability and integrity of signal processing data pipelines is demonstrated in radio astronomy. IBM InfoSphere Streams embraces the stream-computing paradigm. It is a shift from conventional data mining techniques (involving analysis of existing data from databases) towards real-time analytic processing. We discuss using InfoSphere Streams for effective DPDM in radio astronomy and propose a way in which InfoSphere Streams can be utilized for large antennae arrays. We present a case-study: the InfoSphere Streams implementation of an autocorrelating spectrometer, and using this example we discuss the advantages of the stream-computing approach and the utilization of hardware accelerators.
dc.identifier.citationData Provenance and Data Management in eScience Studies in Computational Intelligence Volume 426, 2013, pp 129-156
dc.identifier.doi10.1007/978-3-642-29931-5_6
dc.identifier.urihttps://hdl.handle.net/10292/3141
dc.publisherSpringer
dc.relation.ispartofseriesStudies in Computational Intelligence, Kacprzyk, J; Ditzinger, T
dc.relation.isreplacedby10292/4178
dc.relation.isreplacedbyhttp://hdl.handle.net/10292/4178
dc.rightsAn author may self-archive an author-created version of his/her article on his/her own website and or in his/her institutional repository. He/she may also deposit this version on his/her funder’s or funder’s designated repository at the funder’s request or as a result of a legal obligation, provided it is not made publicly available until 12 months after official publication. He/ she may not use the publisher's PDF version, which is posted on www.springerlink.com, for the purpose of self-archiving or deposit. Furthermore, the author may only post his/her version provided acknowledgement is given to the original source of publication and a link is inserted to the published article on Springer's website. The link must be accompanied by the following text: "The final publication is available at www.springerlink.com”. (Please also see Publisher’s Version and Citation).
dc.rights.accessrightsOpenAccess
dc.subjectStream Computing
dc.subjectRadio Astronomy
dc.subjectInfoSphere Streams
dc.titleData provenance and management in Radio Astronomy: a stream computing approach
dc.typeChapter in Book
pubs.organisational-data/AUT
pubs.organisational-data/AUT/Design & Creative Technologies
pubs.organisational-data/AUT/Design & Creative Technologies/School of Computing & Mathematical Science
pubs.organisational-data/AUT/PBRF Researchers
pubs.organisational-data/AUT/PBRF Researchers/Design & Creative Technologies PBRF Researchers
pubs.organisational-data/AUT/PBRF Researchers/Design & Creative Technologies PBRF Researchers/DCT C & M Mathematical Science
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
dpdmsprg.pdf
Size:
1.29 MB
Format:
Adobe Portable Document Format
Description:
Download
Collections
School of Engineering, Computer and Mathematical Sciences - Te Kura Mātai Pūhanga, Rorohiko, Pāngarau