DeepFake Detection Based on Discrepancies Between Faces and Their Context

Yuval Nirkin, Lior Wolf, Yosi Keller, Tal Hassner

פרסום מחקרי: פרסום בכתב עתמאמרביקורת עמיתים


We propose a method for detecting face swapping and other identity manipulations in single images. Face swapping methods, such as DeepFake, manipulate the face region, aiming to adjust the face to the appearance of its context, while leaving the context unchanged. We show that this modus operandi produces discrepancies between the two regions (e.g., Fig. 1). These discrepancies offer exploitable telltale signs of manipulation. Our approach involves two networks: (i) a face identification network that considers the face region bounded by a tight semantic segmentation, and (ii) a context recognition network that considers the face context (e.g., hair, ears, neck). We describe a method which uses the recognition signals from our two networks to detect such discrepancies, providing a complementary detection signal that improves conventional real versus fake classifiers commonly used for detecting fake images. Our method achieves state of the art results on the FaceForensics++ and Celeb-DF-v2 benchmarks for face manipulation detection, and even generalizes to detect fakes produced by unseen methods.

שפה מקוריתאנגלית
עמודים (מ-עד)6111-6121
מספר עמודים11
כתב עתIEEE Transactions on Pattern Analysis and Machine Intelligence
מספר גיליון10
מזהי עצם דיגיטלי (DOIs)
סטטוס פרסוםפורסם - 1 אוק׳ 2022
פורסם באופן חיצוניכן

הערה ביבליוגרפית

Publisher Copyright:
© 1979-2012 IEEE.

טביעת אצבע

להלן מוצגים תחומי המחקר של הפרסום 'DeepFake Detection Based on Discrepancies Between Faces and Their Context'. יחד הם יוצרים טביעת אצבע ייחודית.

פורמט ציטוט ביבליוגרפי