Reproduce “PerFace: Metric Learning in Perceptual Facial Similarity for Enhanced Face Anonymization” on OpenResearch

About this paper

Haruka Kumagai, Leslie Wöhler, Satoshi Ikehata, Kiyoharu Aizawa

Researchers at The University of Tokyo developed PerFace, a facial feature extractor that accurately quantifies human-perceived facial similarity, specifically for face-swapped images. This model achieved 0.917 accuracy in predicting similarity on a custom dataset, outperforming existing methods, and enhances face anonymization by enabling the selection of perceptually distinct yet natural-looking swapped faces.

Key insights

• Accurate measurement of nuanced perceptual similarity between distinct faces, especially in manipulated images, requires models trained specifically on human-annotated data within that domain.
• Metric learning with triplet loss, applied to a carefully curated human-annotated dataset of face-swapped images, can effectively fine-tune pre-trained face recognition models to capture subjective perceptual similarity.
• High-quality and consistent human annotations significantly impact the performance of models learning perceptual judgments, yielding more reliable and accurate similarity predictions.

Summary by alphaXiv.