Deepfake
lastmod 2023-06-19created 2020-06-097 edits
A deepfake (a portmanteau of deep learning and fake) is a human image synthesis technique based on artificial intelligence. Using a machine learning technique called a generative adversarial network (GAN), it overlays existing photos or videos onto source photos or videos.
Projects
- roop - One-click deepfake (face swap)
Documentation
- DeepFaceDrawing - Deep Generation of Face Images from Sketches
- https://www.reddit.com/r/MachineLearning/comments/gwrmf9/r_deepfacedrawing_generates_photorealistic/
- https://syncedreview.com/2020/06/04/deepfacedrawing-generates-photorealistic-portraits-from-freehand-sketches/
- https://arxiv.org/pdf/2006.01047.pdf
- DeepFaceLab
- https://github.com/iperov/DeepFaceLab
- https://mrdeepfakes.com/forums/thread-guide-deepfacelab-2-0-explained-and-tutorials-recommended
- Unique, worry-free model photos
- https://generated.photos/
- Enhance your creative works with photos generated completely by AI. Find model images through our sorted and tagged app, or integrate images via API.
- Uses deep learning to generate images of people who do not actually exist.
How to detect DeepFake synthesized videos
- rPPG - A novel way to tell DeepFake synthesized videos apart | GeekNews
- (Original) AI researchers use heartbeat detection to identify deepfake videos
- Another take: To Uncover a Deepfake Video Call, Ask the Caller to Turn Sideways - To tell whether a video call is a deepfake, ask the caller to turn their head
Sharing an article (in English) that introduces a novel way to tell DeepFake videos apart. This method distinguishes real human faces from DeepFakes based on heartbeat signals extracted from a person's face.
The technique used to measure human heart rate in non-invasive oxygen saturation monitors at hospitals, or in the now fairly common smartwatches, is technically called PPG (Photoplethysmography). When light is shone on the human body, the light is absorbed, and the degree of absorption is proportional to the volume of skin, tissue, and blood along the path the light travels. The volume of skin or tissue does not change abruptly, but the volume of blood in the vessels at a given location increases when the heart contracts and decreases when the heart relaxes. So if you plot the light absorption pattern as a graph and simply measure the interval between the peaks, that is the heart rate. Simpler than you might think.
rPPG (Remote photoplethysmography) goes one step further. Humans and cameras can see objects because light from somewhere is reflected off the objects and enters the eye or camera. Given the PPG principle described above, the light reflected from body parts such as a living person's face can be said to carry changes that, though very faint, clearly reflect the pulse pattern. Measuring heart rate from nothing but a video of a face recorded by a camera, without any special measuring instruments or contact, by highly amplifying these faint changes is called rPPG. This method relies on extremely subtle changes in facial color, but recent rPPG systems that apply deep learning are known to be able to extract these changes even from fairly heavily compressed face videos.
If so, couldn't extracting biosignals this way also distinguish real human faces from faces made with DeepFake? Papers on experiments that detect DeepFake face videos based on the latest rPPG techniques have recently been published. Put simply, the rPPG signals from real human faces and from faces made with DeepFake have clearly different patterns, and the extracted rPPG signal patterns also differ depending on the type of model used for the DeepFake. As a result, this method could not only tell DeepFake videos apart but also identify which specific model was used with fairly high accuracy.