GAN Generated Images

Let’s play a game - can you guess what these portraits have in common? 

They all depict non-existent people. All these images were created by artificial intelligence. We could say, that AI “dreamed” those people up. 

How can a machine make up people’s faces?

The genius idea came from Ian Goodfellow, who was a PhD student back then in 2014, when he and his colleagues published a paper on this topic. He suggested a concept known as GAN – generative adversary networks. Generative because they generate -> create results (images) and adversary because the two networks act as opponents – they are “playing a game” against each other. One of the networks wants to generate an image which looks realistic, and the other network has the task of telling if the image is generated by the network or if it is in fact an original image from the data set of real photos. 

To better explain this concept we can use the popular forger and critic metaphor, which I personally find very useful. Imagine the following: one of the “players” (the deep learning network) wants to forge paintings, trying to imitate artworks, while the other one acts as an art critic, who he wants to be able to distinguish which painting is an original and which one is a forgery. So the forger begins his job by randomly creating images, and the critic assesses if the image is an original or forged. In our case, original would mean that the image comes from the original data set with portraits of real people. After each turn, both of the players become better at their job, ideally resulting in a situation when the forger creates images so good, that the critic has no way of telling if the painting is real or fake and thus will guess with a success rate of 50%, being right equally often as being wrong.

Want to get more specific?

The first network has the role of a generator – it uses input in the form of randomly sampled noise vector from a normal distribution and creates an output in the form of images, which is then fed to the other network. This one’s role is called discriminator because it tries to tell apart a fake input from a real one. The generator is continuously learning through the feedback of the discriminator, and vice-versa.

Building on this idea, a few years later, NVIDIA, the company you might know for designing and manufacturing graphics processing units, developed the styleGAN algorithm. Also, they made their code open-source in 2019. But to train a GAN, you need very large data sets so that the networks can learn from them. In the case of styleGAN, portraits of celebrities as well as creative commons license photographs from Flickr were used. NVIDIA were so nice and made available also the already trained model so that those interested could skip the time-consuming process of training the networks and jump to generating images right away.

At first, the images created by the styleGAN model resembled something straight out of Lovecraft’s Necronomicon, but be the judge yourself:

Source: https://arxiv.org/pdf/1511.06434.pdf

But as always, progress cannot be stopped and the new styleGAN2 model improved the results significantly, producing the images we fooled you with in the introduction to this article. 

If you would like to try and guess which person exists in the real world and which maybe only in Matrix, go and try at this aptly named page: whichfaceisreal.com 

After you gave it a go, here are some tips on how we can tell them apart (for now) only by looking for visual clues, the so called artifacts. 

How to tell them apart

• Background  

Be on the lookout for weird looking background. Since the whole training of the networks is focused on depicting realistic faces of people, the background generated has a tendency to be suspiciously distorted, blurred, at times it might contain the occasional monster.

• Hair

This has been always a big problem, depicting such an extraordinarily detail-heavy body part like hair is tricky and many times, the image contains detached strains of hair seemingly growing out of uncommon parts of the face or hair looking as if it was painted with a thick brush, not having the appropriate pattern.

• Symmetry

It seems that styleGAN has a hard time with generating pieces that go in pairs, like earrings. So if you see non-matching earrings, eyes looking in different directions, or glasses having mismatching frame shapes it could be a sign of AI doing its business.

• Water droplets

A dead giveaway that you re not looking on a real persons photo is seeing blobs  that look like drops of water. These appear most commonly on parts of the image where the background borders the face.

• Teeth

The styleGAN model has problem when shifting the angle with which a person is facing the image. So at times, the teeth do not shift accordingly, remaining in the same position as if the person was facing the camera directly.

And what if the algorithms get even better?

In 2018 the new styleGAN made improvements, leading to some of those artifacts not happening anymore. Does this mean in the near future we wont be able to tell them apart by simply observing them with naked eye? Probably yes. But worry not, in order for AI to claim its reign and declare us their overlord will still have to fool mathematicians. 

In a simple mathematical analysis of the pixels, images generated by the AI reveal themselves through suspicious patterns. When you convert images created by deep neural network algorithms into a frequency representation, you can easily recognize the because of their grid-resembling pattern. This discovery was made by Joel Frank and his team at the Ruhr-Universität Bochum.

In the discrete cosine transformation, the image is expressed as the sum of many different cosine functions, the researchers explain. While natural images consist mainly of low-frequency functions, artificially generated images show different types of artifacts in the high-frequency range.

It seems to be a structural problem of all deep learning algorithms, explains Frank. "We assume that the artifacts described in our study will always show whether the image is a deep-fake image created with machine learning".

So, you heard the man. There is no need to worry… for now.

Image by Gerd Altmann from Pixabay.