Art

Know Your Language: Meet Processing, the Lingua Franca of Creative Coding

What started as a programming learning tool developed at MIT now pretty much owns code-based digital art.

Michael Byrne

Michael Byrne

Image: ijwthyh

Processing is an enormously popular, very-high-level programming language that began at MIT in 2001. As originally developed by MIT Media Lab alums Casey Reas and Benjamin Fry—both artists and technology thinkers—Processing was intended to be a learning language, a tool to get non-programmers, particularly artists, hooked on code via the promise and delivery of immediate visual feedback.

"Processing began with a simple idea," wrote John Maeda, an artist and computer scientist that advised the duo at MIT, in an excellent history at Technology Review. "Fry was a gifted Carnegie Mellon grad who had a background in graphic design but could also program in Apple II assembly language. Reas was a gifted University of Cincinnati grad who had a background in graphic design and a passionate desire to understand computation; as design director at I/O 360 Digital Design in the mid-1990s, he had been one of the few classically trained graphic designers who understood the possibilities of computer programming, but he left his job (and probably a few gazillion dollars) to study computer code for real at the Media Lab."

Since its early-aughts genesis, Processing has attracted a thriving community of followers and developers, especially among new media types and artists but also well beyond. It has a similarly popular implementation in JavaScript—enabled by HTML5's draw-able, Flash-slaughtering "canvas" element—known just as Processing.js, as well as a soon-to-be-very-popular and still more artist-friendly variation called p5.js. In 2010, Python Mode for Processing was first released and, since 2014, processing.py has been co-developed by Google.

"Processing had two goals when we started in it 2001 and we have these goals today," Reas told me. "We made it to be a flexible software sketchbook and a language for learning how to code within the context of the visual arts. We have always aspired to make the powerful ideas behind creating software accessible for visual artists and designers. We have always promoted software literacy (both reading and writing code) within the visual arts."

"We started the Processing Foundation in 2012 to clarify these ideas," Reas said, "to support the creation of Processing, and to apply these ideas to other domains."

Unlike many if not most of the languages to be featured in this series, Processing is something I would suggest that you, perhaps a non-programmer, go out and try right now because it's stupid-easy and pretty fun. (And I hate fun, usually.) This is the Processing equivalent of "Hello, World": a line.

line(15, 25, 70, 90);

Which is this (show with the IDE in the background):

Processing is also an IDE, or integrated development environment. This is part of what makes it so easy to get into. Download the IDE (for free), write a line of code, and click a button to run it. Whatever you just made—a line, perhaps—will show up in a new window. Congrats, your first computer program, and all it took was about 20 seconds.

Given another 20 seconds and you could be modifying or running one of literally hundreds of code examples and tutorials that come prepackaged with Processing, or one found within the brightly-colored, bottomless chasm of the openprocessing.org community. Processing is learning, but learning also means access to new stuff.

The Processing Foundation

"Processing has always had a priority of access, Reas explained. "As communities of artists and designers, we have aspired and struggled to create our own technologies and more importantly, our own cultures around emerging technologies. Often, we have different priorities and politics from corporations who make software for artists to use. We are a group of artists and designers who make software for ourselves and we share it with others."

It's true enough: As a language for and by artists—as opposed to a general-purpose tool—Processing is in a unique position to be whatever it wants.

"As a larger community, we've done a good job at reducing the barriers to learning and creating with emerging technologies. Our field is fundamentally different from a decade ago in respect to that goal," he said. "However, in relation to diversity, access, and inclusion, we're not doing well enough. The current challenge is to move the environment and culture around technology forward—to improve access to larger and different groups than we've reached before. We plan to achieve this through partnerships."

The Processing basics

Processing code is created in what are known as sketches, which are just files. The Processing IDE is known as a sketchbook and, as far as IDEs go, it's very lightweight, especially compared to monsters like Visual Studio, Xcode, and Eclipse (though Processing is well-suited for Eclipse, as it's Java-based).

The skeleton of a Processing sketch consists of a handful of functions (or methods), which can be viewed as programs within programs. The two biggies are called "setup()" and "draw()." This isn't meant to be a tutorial by any means, but to talk about Processing even in general terms, we also need to talk about these two functions. These are the brain and heartbeat of a Processing program.

Briefly, setup() does just what it sounds like. It creates a window of certain dimensions and with certain properties that will contain everything else the (rendered or live) sketch does. draw() is where the action is. While the initialization function runs once at the beginning of the sketch/program, the drawing function runs continuously, refreshing the window created in setup() at regular intervals given by the frame rate. Just imagine a paper flip-book in digital form.

So, if you were to, say, make a circle bounce up and down, you would write some code that changes the location of the circle on every refresh (or every other refresh or ever third refresh, or whatever). For example, if you had a dot on a screen that started at position x,y, you could write code that increments those values every time the function runs, which is what the refresh actually is: a new instance of the drawing function being called every 1/60th of a second or so. Every time >draw() is called, instead of putting our dot at the location x,y, we put it at x+1,y+1, so the dot appears to moving right (along the positive x-axis) and up (along the positive y-axis).

So, say we start with our point at 0,0, the origin. After 1/60th of a second, it will be at 1,1—one pixel up, one pixel right—and it will keep progressing like that: 1,1;2,2;,3,3;4,4 ...

Here's an actual example from processing.org (a grey ball passing back and forith horizontally):

// Learning Processing // Daniel Shiffman // http://www.learningprocessing.com // Example 5-6: Bouncing Ball int x = 0; int speed = 1; void setup() { size(200,200); smooth(); } void draw() { background(255); // Add the current speed to the x location. x = x + speed; // Remember, || means "or." if ((x > width) || (x < 0)) { // If the object reaches either edge, multiply speed by -1 to turn it around. speed = speed * -1; } // Display circle at x location stroke(0); fill(175); ellipse(x,100,32,32); }

The guts

There is a catch to Processing. It's written in Java. A Processing sketch is like a further abstracted form of a Java program. A lot of the Java guts are hidden by Processing and, given that part of the whole point of Java itself is to hide its own guts in the Java Virtual Machine, Processing can be considered to be a very-high-level language. Between it and the actual computing hardware are a whole lot of layers and additional abstraction, which is not always a good thing when it comes to speed and other things.

Image: joonhyublee

Java is clunky, slow, a product of the Oracle corporation, and as a coding experience, it can kind of feel like eating a TV dinner: simplicity at the cost of everything else. Many people also love it.

Kinect Flock | Processing Concept Sketches Credit: Alex Wolfe

It's an interesting thing—an entire creative galaxy sprouting from Oracle's stuffy Java.

From an FAQ at processing.org:

We didn't set out to make the ultimate language for visual programming, we set out to make something that was: A sketchbook for our own work, simplifying the majority of tasks that we undertake, A teaching environment for that kind of process, and a point of transition to more complicated or difficult languages like full-blown Java or C++ (a gateway drug to more geekier and more difficult things).

Processing is not intended as the ultimate environment or language.

So, why use Java in the first place? There are some pretty good reasons actually. In 2001, Java was finding a lot of use in web-based animation via Applets, which are those small frames you still sometimes see on websites in which Oracle demands that you download the latest version of Java or else. This was more or less the chief resource available to web-based digital creatives, which made it a natural choice.

"In one sense, Processing arose as a response to practical problems," Maeda writes. "When Java first came out, it offered minimal support for sophisticated graphics processing. Drawing a line and stuff like that was possible, of course. But it couldn't do transparency or 3-D, and you were almost guaranteed to see something different on a Windows computer and a Mac; it was incredibly cumbersome to do anything that was both sophisticated and cross-platform."

"So [Ben] Fry," he continues, "who grew up hacking low-level graphics code as a kind of hobby, built from scratch a rendering engine that could make a graphically rendered scene appear the same in a Windows or a Mac environment. It was not just any renderer–it borrowed the best elements of Postscript, OpenGL, and ideas cultivated at the MIT Media Lab in the late Muriel ­Cooper's Visible Language Workshop."

Of course, the programming language world is a whole lot different now and there are any number of ways to do cool stuff on the web or otherwise. Would Reas and Fry have done it differently had Processing been released in 2015?

Maybe.

"We asked [artist and programmer] Lauren McCarthy to explore this question in one of the first [Processing] Foundation fellowships," Reas explains. "How could the initial goals of Processing be achieved in 2014 outside of the technical decisions we made and have been making since 2001? The answer was to apply the ethos of Processing to JavaScript and the web. The p5.js project is the result and she is doing an amazing job as the leader of that project. I think 2015 will be a big year for p5.js."

"We're still developing Processing with Java in 2015," Reas says. "It remains a good balance in how easily ideas can be expressed in code and how quickly the code will run. C++ requires too much detail for the idea of 'sketching.'"

OpenFrameworks

For those unsatisfied with Processing or its brand-name iterations, or for those who prefer programming at somewhat lower levels (meaning: less abstracted from the machine*), there is OpenFrameworks, which I like a whole lot and was my backdoor into Processing-land in the first place).

OF is an open-source "creative coding" toolkit written in C++ providing much of the same capabilities as Processing; it's possible to translate from one to the other pretty easily and many if not most of the functions provided by Processing have suitable analogs in OF. There seems to be a solid crew out there making sure OF is keeping up (at least). For example,. the entire code-base provided by Daniel Shiffman's excellent and popular Processing-based Nature of Code—a user-friendly primer on simulating natural processes algorithmically—has been mirrored in C++ several times thanks to OpenFrameworks. (By the by, Shiffman has his own Learning Processing book and an accompanying website.)

*I know, I know. If it were only so simple.

Arduino

Arduino is just as much of a movement as it is a hardware product, just as Processing is as much a movement as it is a programming language. Its series of microcontrollers—small, inexpensive computers that are capable of programmable input-output operations—has made embedded systems a playground for hobbyists and artists. Like Processing, it's both an entry-level to and a foundation for using technology to do creative things. The standard Arduino IDE looks a whole lot like Processing too:

There's no shortage of projects and libraries marrying Arduino and Processing capabilities—and tutorials explaining how to do so—but we're getting a bit beyond the scope of this post. Just know that it's out there.

As seen in

Getting started

Download the Processing IDE here. It's available for Linux, OSX, and Windows. Next, there are a universe of tutorials to be found at processing.org and beyond. Processing: 0 to "made something cool" in mere seconds.