Social Video Advertisement Replacement and Its Evaluation in Convolutional Neural Networks
Loading...
Files
Size: 756.03 KB, File format: Adobe PDF
Date
Authors
Yang, Cheng
Yu, Xiang
Kumar, Arun
Ali, G. G. Md. Nawaz
Chong, Peter Han Joo
Lam, Patrick
Supervisor
Item type
Degree name
Journal Title
Journal ISSN
Volume Title
Publisher
Universitat Autonoma de Barcelona
Abstract
This paper introduces a method to use deep convolutional neural networks (CNNs) to automatically replace advertisement (AD) photo on social (or self-media) videos and provides the suitable evaluation method to compare different CNNs. An AD photo can replace a picture inside a video. However, if a human being occludes the replaced picture in the original video, the newly pasted AD photo will block the human occluded part. The deep learning algorithm is implemented to segment the human being from the video. The segmented human pixels are then pasted back to the occluded area, so that the AD photo replacement becomes natural and perfect appearance in the video. This process requires the predicted occlusion edge to be closed to the ground truth occlusion edge, so that the AD photo can be occluded naturally. Therefore, this research introduces a curve fitting method to measure the predicted occlusion edge’s error. By using this method, three CNN methods are applied and compared for the AD replacement. They are mask of regions convolutional neural network (Mask RCNN), recurrent network for video object segmentation (ROVS) and DeeplabV3. The experimental results show the comparative segmentation accuracy of the different models and DeeplabV3 shows the best performance.
Description
Keywords
40 Engineering, 46 Information and Computing Sciences, 4008 Electrical Engineering, 4603 Computer Vision and Multimedia Computation, 4611 Machine Learning, Bioengineering, Neurosciences, Networking and Information Technology R&D (NITRD), Machine Learning and Artificial Intelligence, Deep Learning, Image Processing, Image Segmentation, Video Advertisement Replacement
Source
Electronic Letters on Computer Vision and Image Analysis, ISSN: 1577-5097 (Print); 1577-5097 (Online), Universitat Autonoma de Barcelona, 20(1), 117-136. doi: 10.5565/rev/elcvia.1347
Publisher's version
Rights statement
Copyright (c) 2021 Cheng Yang, Xiang Yu, Arun Kumar, G.G. Md. Nawaz Ali, Peter Han Joo Chong, Patrick Lam. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Permanent link
Endorsement
Review
Supplemented By
Referenced By
Creative Commons license
Except where otherwise noted, this item's license is described as Creative Commons Attribution-NonCommercial-NoDerivatives 4.0