Contrasting Big Data Techniques in Exploring New Zealand Road Crash Data

Abstract

Motor vehicle crashes result in high social and economic costs globally and in New Zealand. Therefore, accurate analysis of crash events is critical for evidence-based prevention and policy. This study explored the application of Big Data techniques, specifically Hadoop and MapReduce, to improve the analysis of the impact of weather and speed on motor vehicle crashes in New Zealand. Contemporary Big Data approaches were applied to address the limitations inherent in traditional methods of crash analysis. We used Hadoop’s distributed storage and MapReduce’s processing capabilities on the New Zealand Transport Agency’s Crash Analysis System (CAS) dataset to identify and visualize environmental and spatial trends to a higher degree of understanding. The project involved Elasticsearch and Kibana to make sense of unstructured data in geographic views, while Hue, Hive, and Power BI represented structured data with charts and dashboards. Results show that non-injury crashes, followed by minor crashes, are the most frequent, with over half happening at speed limits between 40–60 km/h. Geographically, Auckland represents crashes five times greater than in the other locations. Strong and extreme weather conditions appear to be a factor in the majority of reported fatal road accidents.