The VICE Channels

    How a Painting Robot Was Given an Artist's Soul

    Written by

    Nadja Sayej


    This artist has painted a handsome portrait of Robert de Niro, knows how to mix the right shade of grey, and can analyze work on the fly. It's also a robot. Meet e-David, a painting bot created by the computer graphics and media design department at the University of Konstanz in Germany, and the first bot of its kind to be able to change and adapt its style in the middle of a painting.

    Oliver Deussen, the IT professor who created e-David, began his work in 2009 by upgrading an automotive welding robot with paintbrushes, a palette, a camera, and a computer. E-David—which stands for "Drawing Apparatus for Vivid Interactive Display"—started with a face sketch of a former university president. Soon after, it was introduced to color and continues to mathematically compose works on paper and canvas, learning styles from stippling to impressionism. Soon, it's expected to learn to paint in the freewheeling style of expressionism.

    Armed with five brushes and 24 colors, e-David applies paint in a polite, detailed manner on the canvas. Like any good student, it washes his own brushes to keep them from crusting and mixes its paint in a nearby palette. Like many other artists, e-David watches what it paints and decides independently where to add more color and what brush strokes to add. Rather than be programmed to copy a painting exactly, the bot views and analyzes its artwork through a camera, and its software calculates where to add color and shading next. 

    Since e-David can only speak in visual metaphors, I spoke with Deussen, who calls e-David “his baby.”

    A Robert de Niro painting by e-David, the painting bot.
    Everybody wants him to paint like Van Gogh. Via Flickr

    Motherboard: Hi Oliver, do you have a background in art? I'm dying to hear how this idea came about back in 2009.

    Deussen: Well, I have been working in the field of non-photorealistic rendering methods for a number of years. With my PhD students, I created imitations of watercolor paintings and other techniques, but everything was just pixel information that had to be printed later. So I thought it would be so nice to use real paint on a real canvas—especially since I paint by myself (without having an art background).

    How is David different from other painting bots, like the BNJMN and the Painting Fool?

    In the e-David project, we want to investigate to what extent the painting process can be described by means of algorithms. We want to mathematically describe different painting styles. Thus we don't have the ambition to create art, but we want to create good paintings from a craftsmanship point of view.

    E-David is not creative; maybe we are creative when programming the algorithms. But in this project, we do not use the machine as an expensive plotter, we do not tell e-David where to apply new strokes. This is the result of a visual optimization process. The machine watches itself while painting and corrects mistakes with the next brush strokes.

    By doing so, we can paint very subtle things and create paintings that look like they’re created by humans. I think this happens because humans also use such a visual optimization process when painting. This visual feedback loop distinguishes e-David from other painting machines like the Painting Fool or BNJMN. It involves quite a lot of image processing, computer vision, and other computer science aspects.

    Can you explain how David’s visual optimization process works?

    After applying a number of brush strokes, we take a high-resolution (rectified, color corrected) photograph of the canvas. We compute the difference between the canvas and the input image that should be painted. We then find regions where color is missing. From our repository, we select the paint that is most similar to the needed color. Since we cannot mix colors on the palette, we have to apply thin layers of color to create the necessary effects. If the program is not able to apply a brush stroke that will further improve the painting, we stop the machine.

    E-David's first work, via the project site

    Can you draw a line between human creativity and artificial intelligence?

    There is quite some theory about human and computer creativity, i.e., by philosopher Margaret Boden. She distinguishes between exploratory and transformational creativity. Exploratory creativity is when you have a style and now you explore how to modify this style to create a slightly new impression. Even a robot might be able to do this kind of creativity in the future.

    Transformational creativity takes a concept from one domain and transforms it into another one; e.g., by incorporating issues of the society into a painting. This will be very hard for robots since it requires a lot of deep knowledge about the world.

    David can paint like an Impressionist. What other kinds of techniques will he soon learn?

    Everybody asks us to create Van Gogh paintings, so we will try to do that next. But also expressionism is very interesting since it requires some deep understanding of the painted object.

    Do the paintings still take 10 hours to complete, or less, since switching to acrylic from black ink?

    No, things are getting even longer sometimes. We are not fast while painting a brush stroke and have to use many different layers of colors until we reach the final state. But the machine does this without complaining too much and also works all night...

    Are there plans for an exhibition of e-David’s works or selling them with a gallery?

    We have some first contacts to museums. Currently, e-David is too big for an exhibition and it would be a nightmare to move it out of the basement of our university building. But we are building a smaller version that can be transported. This version can be displayed almost everywhere and will hopefully be much cheaper.

    We sell individual paintings for 500€ a piece; we have to pay the university when using the machine for commercial purposes, and that's the biggest part of the sum. We would be happy to find a gallery that is specialized on that kind of art, though.

    David even signs his own pictures. Who designed the signature?

    I designed the basic layout, but e-David does an individual signature every time he signs a painting.