HYBRIDJOIN for near-real-time Data Warehousing

Simple item page
aut.researcherNaeem, Muhammad
dc.contributor.authorNaeem, MA
dc.contributor.authorDobbie, G
dc.contributor.authorWeber, G
dc.contributor.editorTaniar, D
dc.date.accessioned2012-04-26T00:42:42Z
dc.date.available2012-04-26T00:42:42Z
dc.date.copyright2011
dc.date.issued2011
dc.description.abstractAn important component of near-real-time data warehouses is the near-real-time integration layer. One important element in near-real-time data integration is the join of a continuous input data stream with a diskbased relation. For high-throughput streams, stream-based algorithms, such as Mesh Join (MESHJOIN), can be used. However, in MESHJOIN the performance of the algorithm is inversely proportional to the size of disk-based relation. The Index Nested Loop Join (INLJ) can be set up so that it processes stream input, and can deal with intermittences in the update stream but it has low throughput. This paper introduces a robust stream-based join algorithm called Hybrid Join (HYBRIDJOIN), which combines the two approaches. A theoretical result shows that HYBRIDJOIN is asymptotically as fast as the fastest of both algorithms. The authors present performance measurements of the implementation. In experiments using synthetic data based on a Zipfian distribution, HYBRIDJOIN performs significantly better for typical parameters of the Zipfian distribution, and in general performs in accordance with the theoretical model while the other two algorithms are unacceptably slow under different settings.
dc.identifier.citationInternational Journal of Data Warehousing and Mining, vol.7(4), pp.21 - 42
dc.identifier.doi10.4018/jdwm.2011100102
dc.identifier.issn1548-3924
dc.identifier.urihttps://hdl.handle.net/10292/4051
dc.publisherIGI Publishers
dc.relation.isreplacedby10292/4176
dc.relation.isreplacedbyhttp://hdl.handle.net/10292/4176
dc.relation.urihttp://dx.doi.org/10.4018/jdwm.2011100102
dc.rightsCopying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
dc.rights.accessrightsOpenAccess
dc.subjectData transformation
dc.subjectData Warehousing
dc.subjectNear-real-time
dc.subjectPerformance and tuning
dc.titleHYBRIDJOIN for near-real-time Data Warehousing
dc.typeJournal Article
pubs.organisational-data/AUT
pubs.organisational-data/AUT/Design & Creative Technologies
pubs.organisational-data/AUT/Design & Creative Technologies/School of Computing & Mathematical Science
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
HYBRIDJOIN-for-Near-Real-Time-Data-Warehousing.pdf
Size:
440.89 KB
Format:
Adobe Portable Document Format
Description:
Download
Collections
School of Engineering, Computer and Mathematical Sciences - Te Kura Mātai Pūhanga, Rorohiko, Pāngarau