A Deep Learning Algorithm for KOL Segmentation on Social Media Videos

Abstract

Nowadays, there is high commercial demand for product replacement which places the products virtually in Key Opinion Leader's (KOL's) social media videos. However, one of the challenges of placing the products virtually is the KOL segmentation. Since KOLs often hold products in front of them, it requires the segmentation to segment not only humans but also different products. This paper introduces the state-of-The-Art deep learning method, namely RSUDISNet, for KOL segmentation. The proposed technique integrates two deep Convolutional Neural Network (CNN) technologies. One is the Matting Objective Decomposition Network (MODNet), which segments KOLs well but not the products blocking the KOLs. The other one is the two-level nested U-structure network (U2Net) based on the salient object detection method to segment the objects well, but not the KOL. The key technique of the proposed research is to employ the feature of the U2Net to embed the MODNet to overcome the problem of KOL segmentation. Since both MODNet and U2Net are lightweights, the combined network can be used for real-Time scenarios. After that, the Intermediate Supervision (IS) training strategy is utilized to overcome the overfitting. The experimental results show that our proposed method outperforms the MODNet and U2Net.