DeepFakes: Deep Issues with Fake Solutions

What are DeepFakes?

DeepFakes are forged or manipulated digital content which are created using deep-learning algorithms. Advances in machine learning and deep neural networks enable anyone, using open source code, to train a model using visual or audio data from a subject and create manipulated content of that subject using their face, body, or voice.

Whether you have only heard about Deepfakes from the Tom Cruise account on Tiktok or if you’ve spent months writing code to create and test detection models, there is almost certainly more to the DeepFake issue for you to learn.

Why am I writing this?

This fall I did a final project for one of my courses on detecting GAN-generated fake face images. While I was able to successfully implement a pipeline for the task, throughout the process I dug through a lot of academic papers on detection and found some logic holes that were never filled. For detecting deep fakes, there seem to be endless models, challenges, and open research questions. However, there doesn’t seem to be much, if any, practicality in the discussion and literature.

Realizing this led me to start researching the issue from other perspectives. Would detection be enough? What are companies doing? What could legislation do? How might we proactively protect videos and pictures?

In general, everything I had read had framed the issue to be a political threat to national security. The biggest risk of deep fakes, in my mind, was spreading misinformation. What I stumbled upon in my search of other approaches shocked me.

While analyzing the threats and possible use cases of deep fakes news articles AND the academic community have failed to address what 96% of deep fakes on the internet are used for: exploitation and humiliation of women. Many academic articles I had read, had one sentence acknowledging the origin of deepfakes on reddit where a user, named “deepfakes” face swapped celebrities onto porn. This sentence was usually followed by several paragraphs describing and analyzing the harm that could be done by a deep fake before an election or through the general decay of public trust.

That’s why I’ve decided to create what I couldn’t find — a framing of the technology that takes into account how it is currently used rather than solely discussing how it could be used. Articles that claim to explain deepfakes to the public must center deep fake porn — which is the first and primary use of the technology. Articles and academic papers which make mentions of solutions must discuss protections and active efforts readers can make as this is an issue that affects more than just politicians. Lastly, academics need to openly address the limitations of their solutions and engage other disciplines to fill the gaps so that practical tools and policies can be created.

So this is what we’ve got here. First I’ll walk you through what deepfakes are — an understanding of the basics will be important for discussing uses and solutions later. Then we’ll dive into real uses. I strive to present a full and accurate picture (the good, the bad and the ugly) in one place. With the context of different uses, we’ll take a deep dive into the technology and process used to make deepfakes where I’ll make some key points about the logistical task and how to protect yourself from being targeted. From there this post will look forward to who this technology is most likely to be used against in the future — spoiler alert, it’s not just politicians! For the last few sections we’ll explore possible solutions. This includes detection models proposed by researchers (and a discussion of their shortcomings) as well as company policies, active legislative efforts, and a brief proposal of how various efforts can and should come together.

Where is this technology actually used?

In 2019 an Amsterdam-based company formerly called DeepTrace (now called Sensity: https://sensity.ai/) conducted an audit to map the landscape of deepfake use online. Their findings were not only shocking but also exposed gaps in both the academic community’s proposed solutions and the media’s coverage of the threat. Such findings, which prompted the research for and writing of this post, are summarized here:

• The number of deepfake videos on the internet is still low (~15,000 in September 2019) but growing at an insanely fast pace (nearly doubling every 6 months)
• This technology’s primary use (96% of videos online) is non-consensual deepfake pornography. I found this point incredibly troubling considering the majority of pop media articles portray DeepFakes as a threat primarily to politicians rather than as a tool used for the humiliation and exploitation of women’s faces.
• Of the deepfake porn on the top 5 websites 100% is of women.

Screenshot of an interactive CNN Business article on DeepFakes. The article focuses primarily of the technologies possible use against male politicians and makes no mention of Deepfake pornography.

• Although the number of deepfake videos is low, the number of views is incredibly high. Looking only at 4 dedicated DeepFake porn sites the audit found over 100 million views of deepfake videos (September 2019).
• Youtube videos make up a small portion of the deep fake videos online but contain a mix of men and women, primarily from the entertainment industry but also some politicians and corporate figures.

The audit is careful to emphasize that the embarrassment and violation of women is the primary use of this technology on the internet. Beyond Deepfake porn videos, apps to faceswap faces onto naked pictures and even “strip” photos of clothed women utilize a similar technology and are developed specifically to target women. This is a point I feel needs to be more strongly addressed by both the academic community and the media in order for solutions to be effective and the problem to be fully understood.

After reading the audit I couldn’t help but wonder why this technology even exists? And since it does — why is it still legal? This prompted a google search of “DeepFake positive uses” with results such as:

• DeepFakes are used for entertainment and art, from political satire to bringing people back from the dead.
• This technology helps translate important video messages into multiple languages such that the voice is that of the speaker and their lips say the right words.
• Additionally, in the medical field GAN generated images can fill out datasets so that models can be trained even with small amounts of real data.
• Similarly, recent research uses GAN generated images to detect bias in models by altering specific features such as skin color or amount of makeup.

While this technology does have many helpful and positive uses, I don’t think we can ignore the harm it is currently doing in the name of “innovation.” We need to find a way to regulate this emerging technology such that development can continue in the right direction and not for the purpose of demeaning women.

How does this technology work?

This description will get pretty technical, if that’s not something you’re interested in, skip to the logistical overview of how people make deep fakes now.

Deepfake technology gets its name from the use of deep neural networks. If you aren’t familiar with neural networks you can learn more here, but what’s most important for you to understand is that deep neural networks are known for their ability to represent complex and high dimensional data. Specifically, deep autoencoders are used for image compression with which visual or high-dimensional data can be reduced to a set of features and stored as a series of numbers. When paired with a decoder, the vector of numbers can be used to reconstruct the original image.

The first part of generating a deepfake is training such an encoder-decoder pair. The network is given an image, transforms it into a vector, then reconstructs the image. After each iteration the loss is calculated by comparing the regenerated image with the original and used to improve the network through a process called backpropagation. As you can see in the graphic below, both people have their own decoders which learn to accurately transform a vector of features into that person with a specific orientation and facial expression. The encoder, however, is shared by both targets so that it learns to represent facial expressions, orientation, and other features in the same way for both people.

Once training is complete, features are encoded the same way for both people, so an image of one person can be encoded and then be mapped onto the other person by using their decoder. This results in the creation of realistic looking mannerisms and facial expressions for whichever person is desired. This process is not limited to face swapping and can also be used to manipulate body movements, lip sync a video to a specific audio, or generate speech with someone else’s voice.

Graphics from https://arxiv.org/pdf/1909.11573.pdf

By adding adversarial loss and perpetual loss to the encoder-decoder architecture, an improved method based on general adversarial networks was created and is known by Faceswap-GAN (Code available at https://github.com/shaoanlu/faceswapGAN). In this architecture, depicted below, the mismatched encoder-decoder pair is used to generate faked pictures which are then sent to a discriminator which predicts whether the image is real or fake. The different forms of loss calculated throughout the process are used to update weights throughout the network and improve both the generation of realistic looking fake photos and the classification by the discriminator. In this way, the generator continuously gets better at making the photos it generates more realistic and less easy for the discriminator (and humans) to detect as fake.

Graphic from this paper

Later, StyleGAN[8] and other architectures were proposed as a mechanism for creating completely artificial faces.

Logistical Overview

There are also a few logistic notes about the process of making deep fakes that I want to mention.

• Deep Fakes require a lot of data. Training the generator requires thousands of images for both subjects (in diverse orientations and expressions) to create a realistic looking fake.
• However, as the process is improving this number is becoming smaller with some online services only requiring 250 photos which could easily be grabbed from an Instagram, facebook, or youtube account (make sure your accounts are private).
• Creating a deep fake requires a lot of time. Again, this is changing as the process improves and more powerful GPU’s become available through cloud computing services.
• There are many online communities dedicated to the creation of deep fakes. DeepTrace’s audit from 2019 found 20 such communities with in total likely over 100,000 users.
• There are many popular open-source libraries on github and apps you can download to your computer that enable anyone to utilize this technology for free.
• There are marketplace services where people can pay to have a deep fake of a specific target generated for them at varying but inexpensive prices.

As this process continues to become easier, quicker, and cheaper it is more likely to be increasingly used against everyday people, therefore it is important to take extra measures to protect your photos and videos to stop malicious actors from taking advantage of your data.

Who is at risk?

As discussed at the start of this post, this is the primary question asked and partially answered in news articles about DeepFakes. I include it only to highlight that it has two sides, one of which is too often ignored. By addressing the “possible” uses of deepfakes and speaking about the “threat” this technology presents, we overlook the harm it is currently doing and the women it has already been leveraged against. Countless articles on the internet, like the CNN business article above, which are written to expose the public to the issue make claims about the “real threat” and “most dangerous use” but make no mention of deepfake pornography and the devastating effects on women’s lives.

Various headlines from articles which claim to expose the risks of deepfakes but make no mention of deepfake porn.(Top Left; Top Right; Bottom Left; Bottom Right).

We’ve spent too much time creating and analyzing the damage it could cause in speculative situations and too little time addressing the abuse that has been allowed in thousands of videos and images against the group this technology was created to target: women. Have we decided that women being targeted, harassed, and silenced is okay? Are the consequences to their career, reputation, and emotional well-being insignificant? Is the first and primary use of this technology not a “real threat”?

Women

“Punished for being a woman with a public voice of any kind.”

In answering who is at risk all sources should first and foremost assert that women are most at risk and are least protected. As legal scholar Danielle Citron has written and spoken about, the online harassment of women is nothing new and deep fakes are just the most recent and technologically advanced means of achieving the same ends. However, now the whole world can watch a woman do sexual acts she never did, objectifying her in a way that is hard to repair and resulting in extreme societal consequences for choices she never made.

One example Citron explores in an interview with the Intelligencer is the case of Rana Ayyub, an Indian journalist who writes about government abuses and human rights violations especially for Muslim minorities. A deepfake pornographic video rapidly spread throughout India through WhatsApp groups and resulted in online harassment by masses as well as threats of rape and murder. She stopped writing and it is unlikely she will be able to continue her career. She is experiencing drastic, life-altering consequences for something she never did.

Another case discussed in an article from MIT Technology review is of a poet and broadcaster in the UK who was the victim of a fake pornography campaign created with images from her social media. She felt “Punished for being a woman with a public voice of any kind.” To make matters worse, in attempting to address the issue and get the videos and images removed she found no help from law enforcement. Victims like Mort often find themselves without any recourse or ability to get the media removed from the sites where they are published resulting in spread of the media and more drastic effects the longer it is left up.

Political Figures

While a seriously damaging and widely believed deepfake campaign has not yet taken place against any political leaders, many scholars and journalists have extensively described how severe and powerful such a campaign could be. So far many of the videos which have been generated outside of deepfake porn are for satire, to promote a specific organization, or to demonstrate the power of the technology. However, by studying the use of shallow fakes (real videos manipulated in various ways) such as this video of Nancy Pelosi and fake images such as the one below of Parkland shooting survivor Emma Gonzalez ripping the constitution, we can understand the damage a realistic looking fake political video may have.

Graphic from https://scholarship.law.bu.edu/cgi/viewcontent.cgi?article=1640&context=faculty_scholarship

The Public

Additionally, we are all at risk, not just of being victims, but of supporting deception and of being deceived. An MIT study found that fake media and information are more likely to be shared as demonstrated through the finding that tweets with misinformation “ reach their first 1,500 people six times faster.” Further, this risk of deception isn’t solely surrounding the potential for fake media to damage our perception or understanding of reality, but also the potential for doubt raised about a real video or audio clip to have the same effect. Scholars call this decay of trust in real evidence the liar’s dividend. It is easy to imagine how videos exposing incidents of police brutality or audio clips exposing a President’s sexually derogatory comments could be subject to baseless skepticism and disbelief due to ungrounded claims of forgery. For this reason, it is absolutely necessary that the public has a general understanding of what is possible and what isn’t in this technological arena.

What are scientists doing about this?

There are a few avenues that researchers are taking to address this issue and develop solutions. The most common is developing effective models for detecting manipulation in photos, videos, and audio.

Detection

Some of these detection methods utilize visual artifacts and physical irregularities which everyday media consumers can also use to identify manipulated videos. Some things to look for:

• Deepfake videos lack reasonable eye blinking
• Deepfake videos often have incoherent head poses
• Deepfake videos often lack reasonable eyebrow movements
• Fake videos and images may have visual artifacts such as discoloration around the face
• GAN generated synthetic faces often have errors in accessories such as earrings or glasses
• GAN generated synthetic faces often have visual artifacts in the hair or background

Researchers often use Convolutional Neural Networks (CNNs) trained on a combination of real and fake videos or pictures to produce a binary classification of real or fake for a given input. However, there are several challenges associated with this task.

Challenge 1: Generalizability

One existing challenge is the need for generalizability. Models which are able to detect one form of deepfake, say face swapping, often do not perform well of deepfakes with lip manipulation. Similarly, models which perform well for detecting fake photos do not perform well for detecting fake videos and vice versa. Because of this, it is unclear how the models many researchers propose which perform well for one task could be used in a setting where there might be multiple types of deepfakes. One group of researcher is tackling the generalization issue by creating a pre-processing module named AMTEN which highlights manipulation traces for various types of manipulation. Pairing AMTEN with a CNN evaluated on a dataset with various types of fake face images, the group achieved an average accuracy of 98.52%. AMTEN may be a possible solution for detecting fake face images of different types, but it is unlikely to work for videos and may not detect fake images generated by a new method which wasn’t present in the dataset. When considering how fast the technology is changing and how often new manipulation methods are created it quickly becomes clear why the generalization task will continue to be a challenge.

Challenge 2: Post-Processing

Further, when we consider the case of fake media being posted on social networks or shared through messaging apps the detection task becomes complicated by the post-processing manipulations which take place while uploading or sending media. Upon upload media is often compressed so that it can be stored using a small amount of data. Additionally, upon upload images and videos are often cropped or rescaled. Such alterations can result in visual artifacts or other indicators of manipulation being lost, making detection significantly less accurate. a study by Marra et al. explored detecting GAN generated images on social media, and found that a twitter-like compression of target images decreased average classification accuracy of the highest performing model from 94.49% to 87.17%. If models proposed in research papers are to take seriously the task of detecting fake media in real settings, they must consistently look at the impact such processing has their methods.

Challenge 3: Realistic data sets

Another issue limiting the effectiveness and applicability of detection models is the limited amount of data. Although new datasets have been released in recent years, the majority still contain deepfakes with only one type of manipulation and subjects in a specific position. The datasets that have been created are fit for the task of detecting a deepfake of an individual talking in close proximity to the camera, as might be the case in a video used for political misinformation. However, these datasets (and therefore all models trained using them) have little to no applicability to detecting deepfake pornography or faked naked images of women. This isn’t surprising considering the corresponding papers for two popular datasets in this area, FaceForensics++ and Celeb-DF, mention many of the risks and harms associated with deepfakes but fail to make any mention of the technology’s origin and primary use: deepfake porn.

So although there are hundreds of published papers creating new methods for and improving the accuracy of deepfake detection, these projects do so largely without considering the real problem and what role their solutions could play in addressing these problems, if any.

What are companies doing about this?

Due to the framing of the issue in news and academic research, far more effort has been made to prepare for and prevent political deep fakes than pornographic deep fakes. This gap is apparent when looking at different companies’ policies and engagement of the issue.

Porn Companies

In February of 2018 PornHub claimed to ban deepfakes under their rules regarding non-consensual pornography. However, all their ban accomplished was banning the term deepfake from titles and searches and removing a few channels which exclusively produced fake content. As this Vice article demonstrates deepfake pornography is still present on the site and can easily be found by searching various celebrity names such as Ariana Grande or Selena Gomez who have never been involved in porn. Despite PornHub’s own statistics demonstrating how popular female celebrity searches are and several deepfake videos having millions of views this is unlikely to change even though it violates PornHub’s own policy.

It gets worse! Other porn sites such as XVideos and Xnxx which consistently rank in the top 10 most visited websites on the internet (more visits than amazon) have no policies against deepfake content resulting in videos with extremely high viewership such as one that uses Emma Watson’s face which has more than 23 million views.

Tech Companies

Outside of porn websites, tech and social media companies have experienced lots of external pressure to prepare for the use of deepfakes to spread misinformation. To address this issue Google released a dataset and Facebook released a dataset in conjunction with a detection challenge offering a 1 million dollar reward for the most accurate model. However both of these datasets and corresponding papers fail to address the issue of deepfake pornography and how their efforts contibute (or don’t) to solving that issue.

Twitter and other social media companies have released policies which involve censoring content that meet certain criteria. Here are the questions twitter uses to determine whether tweets should be removed:

• Is the media synthetic or manipulated?
• Was it shared in a deceptive manner?
• Is it likely the content will cause serious harm?

However to be evaluated tweets require being reported and to be censored they must be evaluated, both of which take place while the media can continue spreading. Therefore detecting and censoring in real time can be a very difficult task.

Regardless, much more thought, effort, and money has been put towards detecting deep fakes (specifically political deep fakes) by tech companies. In the case a deepfake with misinformation were to be posted on social media, tech companies will have policies in place for handling the situation. On the other hand, porn websites have made little to no effort to address the issue although it causes serious reputational and emotional damage to real women.

Will these efforts be enough?

In short, no. Neither detection nor individual company policies will be enough to fully address this issue and to stop people from using deepfake creation technology specifically to harass women. Beyond how the detection models which have been developed are not fit for the task of detecting fake porn (due to lack of focus on generalization and absence of applicable datasets), detection alone won’t be sufficient for any type of deepfake. In this article, researchers as UC San Diego try to deceive the best performing detection models using adversarial deepfakes. In a white box setting, where information about the detection pipeline is known, they were able to achieve an average success rate of 99.85% for fooling XceptionNet and 98.15% for MesoNet. In a black-box setting, where information is not known about the detection pipeline, they were able to generate adversarial videos with an average success rate of 97.04% for XceptionNet and 86.70% for MesoNet. This paper raises another challenge for the field of detection: defending the best models from deception. Along with generalizability, the constantly changing techniques, and the variation in post processing manipulations which take place it seems detection is unlikely to be effective in any setting.

Even if it were possible to create a detector that works for many types of fakes, is constantly updated when new ones arise on the internet, does not fail when there are post-processing manipulations, and is able to defend against adversarial inputs, that probably still wouldn’t be enough. Would this detector be used by companies when there is no real incentive to censor content that isn’t explicitly illegal? If a detector claimed a realistic looking video is fake would the public believe it? Would that correct the damage done? Would the public believe when a detector says a video is real? How do we handle the possibility for error?

Detection will not be enough unless other measures are taken to decrease the amount of harmful deepfakes that are produced and increase the consequences for people who create malicious content with this technology. Censorship and bans in individual companies will just cause the content to migrate elsewhere, as is the case for deep fake porn where there are now lots of popular websites dedicated solely to this type of content. Even if a video is reported and removed, the creator can easily create a new account or repost it on a different site.

What else can be done?

Legislation will be necessary in conjunction with technical tools for fighting malicious uses of deepfake technology. The UK has made efforts to address this issue. Currently there are no countries which ban fake nonconsensual porn on the national level. The US and UK have laws prohibiting non consensual illicit content like revenge porn, however only 2 US states (Virginia and California) have laws which include fake and deepfaked nonconcensual content. According to an article from February 2021, the UK Law Commission which reviews laws and recommends reforms is currently working with a deepfaked porn activist while reviewing laws related to online abuse. This suggests regulation of this technology may soon take place on a national level in the UK, hopefully leading the way for the US to do the same. A main reason efforts have failed on the national level in the US are claims that proposed laws overreach and therefore violate the first amendment. However, due primarily due to legislators concerns over deepfakes possible political uses, there is energy for passing meaningful legislation which will hopefully include bans for deepfake pornography soon.

Hopefully legislation will serve as a deterrent factor and means of legal recourse for victims. Until then we can all take measures to protect ourselves such as ensuring accounts are private and not accepting follow or friend requests from strangers. Additionally, researchers from the University of Albany suggest that applying adversarial perturbations to your photos might be one way to ensure your face isn’t used for purposes you don’t choose. Adversarial perturbations add hidden signals to photos that distract face detectors without being perceptible to human eyes. This causes automated programs which scrape profiles and webpages

for face images to produce datasets polluted with images that are not of faces as shown below.

Graphic from https://arxiv.org/pdf/2003.09234.pdf

Other researchers have suggested taking a different approach to the problem by proving authenticity of media rather than detecting fakes. One possible method of doing so would be creating a digital ledge for each newly created media. Blockchain and other digital ledger technologies create community accountability by storing an inalterable ledger which includes the creation and all alterations to the object. This would enable us to track down sources for various media and effectively prove or disprove authenticity. However, use of digital ledger technologies for digital provenance would require buy in from several different parties which may not be realistic in the near future.

But the most important way to fight deep fakes and prevent the technology from being used to harass women or erode public trust in media is to educate yourself and those around you. Knowing about what deepfakes are can help people avoid being deceived. Understanding liars dividend and how people in power may use public skepticism for their advantage will help us keep our leaders and institutions accountable. Lastly, acknowledging the primary use of this technology as humiliating women will help us create and push for policies that help the most at risk population.