Data mining techniques for analysing the weather patterns in Kumeu multi-sensor data

Abstract

The paper investigates an approach consisting of data mining techniques namely, rule extraction and prediction, to finding any existing interesting patterns not possible by other means, embedded in meteorological data acquired by sensors installed in Kumeu, West Auckland, New Zealand, by Geoinformatics Research Centre (GRC). In this research, using processes of the Cross Industry Standard Process for Data Mining (CRISP-DM) model (1), data relating to weather conditions collected over a four year period (from Nov 2008 to Dec 2012) is analysed, the DM processes used being business/problem domain understanding, data understanding, data preparation, modelling, evaluation and deployment. Initially the raw data (141759) extracted from a web database is cleaned and pre-processed following which DM functions, such as C5.0, CRT, CHAID, QUEST and artificial neural network (Clementine software package) were run to find rules and primary attributing variables relating to the weather conditions/ predictors for gusts or gust classes (low, medium, high and very high). From the initial results, the main contributing variable for gust appears to be wind speed, followed by others. Finally, regression and PCA were run to verify the conditions/predictors established by DM techniques. The weather patterns found through this initial investigation are specific to the location and can be further analysed to predict extreme weather events, such as frost, rainfall, humidity that are useful for the local farming communities in field management, e.g. Fungicide spraying