Draw a cat as a black-and-white line drawing – and this site turns it into something nightmares are made of.
Image-to-image translation in Tensorflow is the technical name, but basically it’s a model that works by training pairs of images such as these cats, and then attempts to generate a corresponding output image from any input image you give it. pix-2pixTensorflow was originally available on GitHub changing maps to aerial and day to night within images, for example. This is where you can download Christopher's modified model.
The demo works to transform the input image to get an output image using a generator and discriminator. Christopher discovered the generator and discriminator could be trained to create an output picture of your drawing – what he calls a Tensorflow port.
That all sounds very confusing, but essentially we love it because you can draw silly cats (sometimes unintentionally nightmarish) and it’ll create a more realistic version for you, as you can see in Julian Glander’s attempt below.
The model is trained on around 2,000 cat stock photos and also thousands of shoe and handbag stock photos.
Christopher admits himself how creepy some of the images can turn out.
"I think because it's easier to notice when an animal looks wrong, especially around the eyes. The auto-detected edges are not very good and in many cases didn't detect the cat's eyes, making it a bit worse for training the image translation model," he says.
You can read about his process of the image-to-image demo here.