This AI Learned How to Design Mario Levels by Watching YouTube

We've seen several computers recently that can teach themselves how to play Super Mario Bros. It's cool the first time, but the second time? Big whoop. What other amazing/frightening developments in artificial intelligence do you have for us today, science?

How about a computer that can teach itself to design Super Mario Bros. levels by watching other people play the game on YouTube and Twitch? That's sufficiently mind-blowing, and exactly what Georgia Institute of Technology researchers have developed.

The system focuses on the level (as opposed to Mario) and the positioning between elements on screen like pipes, blocks, coins, and Goombas, detecting the spots where players spend the most time, and determining the relationships between elements to understand level design rules.

For example, after watching players on YouTube, the system understood that the green pipes need to jutt out of the ground, and how to create obstacles that aren't impossible for Mario to jump over.

"We have a number of goals," associate professor of Interactive Computing Mark Riedl told me. "In the long term, we hope to learn something about computational creativity, the question of the extent to which intelligent systems can create like humans. We chose computer games as the focus of our computational creativity research because games are functional artifacts—they must work, they must be playable, there is a lot known about good game design principles. This provides objective means of measuring our progress."

A second goal, Riedl said, is to explore ways in which AI can help humans with little experience create games. Making games is hard, requiring proficiency in both programming and game design, but interest in games has never been higher. Riedl and the primary researcher on the computer generated level design work, Matthew Guzdial, hypothesize that aspiring designers could collaborate with an AI that has the goal of making the human a more effective creator. In order for that system to work, it first has to learn how to make levels itself.

At the moment, the system can only create sections of levels, and none of these have been ported to a playable form yet, but Guzdial says they look fun. As you can see, the images of level segments the system generated so far kind of look like the real thing.

Levels will also get more interesting the more games the system absorbs. One of the next steps, Riedl said, is to train the system on multiple games of the same genre. For example, if it watched a bunch of Sonic games, it'll be able to mix, match, and merge rules from different games and create more interesting designs.

There's going to be a lot on new, 2D Mario levels coming at us soon. At E3 this year, Nintendo showed off Super Mario Maker, a game that allows players to create and share new levels. It looks brilliant.

The difference between user-generated levels and computer-generated levels, I thought, is that an AI could maybe learn to emulate existing game design tropes, while a player could create new, novel design tropes that tweak and subvert what you'd expect. That's a huge part of what makes Mario, and any other game, so fun: continually learning new rules.

Guzdial explained that they're tackling this issue, as well.

"Its actually producing novel content now!" he said. "In terms at least of structures that don't exist in the original Mario levels. For example there are areas in the original Super Mario Bros. with lots of 'trees' Mario jumps on. One of the sections of levels from our system included a 'tree' with a 'branch,' something that didn't exist in the original Mario game, but which could exist based on the rules it learned."

This isn't really the type of novelty that's been convincing players to buy Mario games for 30 years, but it shows the potential, and Guzdial said the next steps in their research should directly touch on this issue.