Deep Learning Algorithm for Virtual Advertisement (AD) Replacement on Fence Board of the Sports Games Videos

Abstract

In the rapidly evolving field of sports broadcasting, the demand for effective and dynamic advertisement placement has increased dramatically. This thesis presents a deep learning-based solution for virtual advertisement replacement on fence boards in sports videos. Our approach leverages Computer Vision (CV) techniques to provide automatic and precise ad placement, even in the presence of common obstacles such as occlusion.

The methodology consists of three main stages. In the first stage, we employ a Key-point Region-Based Convolutional Neural Network (Key-point RCNN) model for fence board detection in the video frames, which is trained and fine-tuned using a manually labelled dataset. This enables the model to identify the optimal locations for placing advertisements (Ads). However, object occlusion could affect the Ads' visibility and effectiveness, especially by humans. We introduce a solution in the second stage using a Matting Objective Decomposition Network (MODNet) model to handle this challenge. It is used to segment the human element from the video frames and re-apple them after the Ads are placed on the fence boards. In the final stage, FlowNet is utilized to monitor the placement of Ads. This ensures that when the scene transitions, the new scene is promptly detected, and ad placement is deferred until the fence board or the optimal location, becomes visible.

Our results indicate that this approach achieves high accuracy in fence board detection and human occlusion handling, overcoming the limitations of previous techniques. The proposed solution opens new possibilities in virtual advertisement replacement in sports videos, presenting a promising avenue for further research and practical applications.