Exploiting Human Face Segmentation for Improving Portrait Image Style Transfer

This research proposes a model to address the common issues of facial distortion and loss of details in style transfer of images containing faces. The model aims to achieve background stylization while preserving facial integrity and improving image clarity by reducing potential blurring during the style transfer process. To achieve this goal, firstly, a pre-trained CycleGAN model is used to perform Monet-style transformation on input images, with a focus on background stylization. Simultaneously, a pre-trained GFPGAN model is employed to enhance the clarity of facial regions in the images. The EasyPortrait dataset is used for model training to find the optimal parameter settings that minimize the loss function, resulting in the lightweight facial segmentation model. and the performance of the pre-trained Segment Anything Model (SAM) model is compared with the self-trained lightweight EasyPortrait model. Based on the evaluation results, the model with superior performance is chosen as the primary facial segmentation model. Subsequently, the high-resolution facial images obtained are fed into the facial segmentation model. Finally, the stylized facial regions are replaced with high-resolution facial images, achieving background stylization and facial enhancement in the original images.