TY - JOUR

T1 - Mask-FPAN

T2 - Semi-supervised face parsing in the wild with de-occlusion and UV GAN

AU - Li, Lei

AU - Zhang, Tianfang

AU - Kang, Zhongfeng

AU - Jiang, Xikun

N1 - Publisher Copyright: © 2023 Elsevier Ltd

PY - 2023

Y1 - 2023

N2 - The field of fine-grained semantic segmentation for a person's face and head, which includes identifying facial parts and head components, has made significant progress in recent years. However, this task remains challenging due to the difficulty of considering ambiguous occlusions and large pose variations. To address these difficulties, we propose a new framework called Mask-FPAN. Our framework includes a de-occlusion module that learns to parse occluded faces in a semi-supervised manner, taking into account face landmark localization, face occlusion estimations, and detected head poses. Additionally, we improve the robustness of 2D face parsing by combining a 3D morphable face model with the UV GAN. We also introduce two new datasets, named FaceOccMask-HQ and CelebAMaskOcc-HQ, to aid in face parsing work. Our proposed Mask-FPAN framework successfully addresses the challenge of face parsing in the wild and achieves significant performance improvements, with a mIoU increase from 0.7353 to 0.9013 compared to the current state-of-the-art on challenging face datasets.

AB - The field of fine-grained semantic segmentation for a person's face and head, which includes identifying facial parts and head components, has made significant progress in recent years. However, this task remains challenging due to the difficulty of considering ambiguous occlusions and large pose variations. To address these difficulties, we propose a new framework called Mask-FPAN. Our framework includes a de-occlusion module that learns to parse occluded faces in a semi-supervised manner, taking into account face landmark localization, face occlusion estimations, and detected head poses. Additionally, we improve the robustness of 2D face parsing by combining a 3D morphable face model with the UV GAN. We also introduce two new datasets, named FaceOccMask-HQ and CelebAMaskOcc-HQ, to aid in face parsing work. Our proposed Mask-FPAN framework successfully addresses the challenge of face parsing in the wild and achieves significant performance improvements, with a mIoU increase from 0.7353 to 0.9013 compared to the current state-of-the-art on challenging face datasets.

KW - 3D face

KW - Face analysis

KW - Face landmark

KW - Face parsing

KW - Generative adversarial network

U2 - 10.1016/j.cag.2023.08.003

DO - 10.1016/j.cag.2023.08.003

M3 - Journal article

AN - SCOPUS:85168795739

VL - 116

SP - 185

EP - 193

JO - Computers and Graphics

JF - Computers and Graphics

SN - 0097-8493

ER -