Kiwifruit Detection and Tracking from A Deep Learning Perspective Using Digital Videos

Abstract

With the growing popularity of ChatGPT, deep learning is rapidly advancing, leading to the development of new techniques and applications in various domains, including agriculture. Fruit detection, tracking, and counting play vital roles in crop management and yield prediction in the agricultural automation industry. However, conventional machine learning methods rely on manual inspection and are labor-intensive and prone to errors. In contrast, deep learning-based visual object detection and tracking algorithms have gained attention for their potential to improve the accuracy and speed of fruit detection and counting.

In this thesis, we propose a novel approach for kiwifruit counting in videos that integrates state-of-the-art models with Kalman filter algorithm. Our method leverages the visual object detection capability of the improved YOLOv8 model to identify and locate individual kiwifruits in images, while the Kalman filter tracks their position and trajectory over time, even when partially occluded or obscured by other objects. Duplicate counting is reduced using the Hungarian algorithm for matching.

We evaluate the effectiveness of our approach on a dataset of kiwifruit images and videos for training and performance assessment. Our results show that the proposed approach outperforms to the existing methods in terms of accuracy and robustness in detecting, tracking, and counting kiwifruits. Our kiwifruit detection module achieved a mean average precision at intersection over union of 95.6%, after combined with the kiwifruit tracking and counting module, resulted in an average counting accuracy of 0.782. Our research contributions include labeling a practical kiwifruit dataset, implementing attention mechanisms and modifying the IoU in the detection model to improve fruit detection accuracy, and enhancing the yield prediction model through the integration of a Kalman filter tracking model for kiwifruit counting.