Dartmouth Is Holding a Turing Test Competition for Poetry, Prose, and Beats

Chatting with robots is old hat, why not get fooled by a robo-Proust or an AI Diplo? Dartmouth's finally giving us the Turing championship we need.

The university's Neukom Institute for Computational Science will be holding three Turing Tests in Creativity that will benchmark the "humanness" of machine-generated poetry, literature and, interestingly, DJ sets. The competition will be held throughout the next academic year and will be open to contestants around the world.

The competition is based off of British computer scientist Alan Turing's proposal for an experiment to determine machine intelligence, i.e. if it could convince people that its output was from a human. That became the Turing test, and the idea's been applied to everything from chatbots to imitating Reddit users to recognizing things and people in photos.

"Turing was deeply interested in the degree to which in replicating mind and machine. Now we such a multidimensional view of intelligence, and holding conversation was just one piece of it," Daniel Rockmore, a professor of computer science and director at Neukom, told me.

Here's a rundown of the three competitions:

DigiLit will have contestants generate short stories that'll hopefully be up to par with short story selections from the New Yorker or a college's MFA program. It'll be akin to NaNoGenMo, a contest for writing a program that successfully outputs a comprehensible short story. But this one with a cash prize of $5,000 if someone's machine-generated short story convinces a majority of judges.

PoetiX is very similar, but with sonnets instead of short stories. Both DigiLit and PoetiX submissions will be judged for humanness and will reward $3,000 to runners-up.

AlgoRhythms will hold an actual two-floor dance party for judges. Every 15 minutes, booths will switch out for a human or a machine DJ. The DJs will pull from a pre-approved set of 10,000 open-sourced songs. Audiences will be prompted by phone or polling card to choose whether they think the set was made by a human or machine DJ. The team behind the machine DJ that gets the most "human" votes will win $3,000.

Rockmore said he got the inspiration for the contests while thinking about tracks in a spin class.

"I wondered if there was technology out there that was creating the set for a spin class. The instructor tries to create a sequence of music with different sorts of cadences, so that the class could match it in some way," he explained.

He thought the tracks could have been chosen through software, but really, it was all done by a human listening to the tracks.

He said that AlgoRhythms had the best shot at succeeding due to the wealth of data already out there on people's music tastes. In fact, we've already got a site that can tell you if a song's a banger, so really, this contest's half over. The other two competitions are facing more tangled problems—syntax, mood, cadence, everything has to be coded so that a machine can recognize what makes prose work.

"What is the distance between man and machine in this particular venue of intelligence?" Prof. Rockmore asked. "Anybody taking the test seriously would know you'd learn all sorts of new things just articulating how these creative acts take place."

Rockmore dropped a mention of Her, the Spike Jonze film that showed a potential future where an artificially intelligent being could love. But what was interesting to him was that robots didn't take every job—notably they couldn't write love letters the way the protagonist could.

So why not start here and now?