How are Deepfakes Made?

Last week, I introduced deepfakes and provided a few examples of deepfake media. This week, I want to discuss how deepfakes are made.

Deepfake media (DFM) are made through two main processes that contain coded algorithms and machine learning. For the sake of clarity, the goal of this hypothetical deepfake is to put Selena Gomez’s face on my body. The first process contains two AI algorithms: one encoder and one decoder. The encoder’s job is to compare thousands of pictures of my face with thousands of pictures of Selena Gomez’s face. The encoder will recognize and identify the similarities between my face and Gomez’s, and the pictures of our faces will be reduced to our common features while still remaining separate from each other (i.e., there will be one set of images of my face and another set of Gomez’s face. The images will just look very similar to each other). Then, the decoder is trained to recognize my face through facial recognition algorithms. In order to place Gomez’s face on my body, the decoder is fed the reduced images of Gomez; then, the decoder reconstructs the images of Gomez onto the structure of my face and is trained with my facial expressions.

The second process for creating DFM is known as general adversarial networks (GANs). Unlike the first process, GANs are used to create entirely new faces versus swapping real faces. GANs use two AI algorithms; one algorithm is trained to make new faces, and the other is trained to recognize fake faces among groups of real faces. The first algorithm is trained with a set of criteria for creating faces (e.g., two eyes, one nose, two lips, two ears) to ensure that the DFM is as realistic as possible. The algorithms enter a loop: the first algorithm creates a face and sends it to the second algorithm with pictures of real faces; the second algorithm identifies the fake face and gives feedback to the first algorithm; the first algorithm uses the feedback to modify the fake face and sends it back to the second algorithm. This cycle continues until the second algorithm can no longer identify the DFM versus the real media.

As technology progresses, deepfake media is able to become more and more realistic, and it’s getting harder to distinguish between real versus fake images. Next week, I will discuss how we currently identify deepfakes.