DARPA Uses Preteen Gamers to Beta Test Tomorrow's Military Software
Inside the nondescript Pentagon-funded research lab developing games under the pretense of teaching elementary students STEM skills.
Sieg Hall doesn't look like much from the outside. Located at the University of Washington, the building was constructed in the 1960s, when it was a focal point for Vietnam-era antiwar protests. Before renovations were carried out it had become so dilapidated that students had a tradition of taking home chunks of rock off its façade. If I didn't know better, Sieg is just another nondescript computer science building, not a front line in military research and development.
But it's here, tucked away on the third floor, that you'll find precisely that: the Center for Game Science, a research lab that makes educational video games for children, and that received the bulk of its funding from the Defense Advanced Research Projects Agency, the wing of the US Department of Defense that supports research into experimental military technology.
Why is DARPA the original primary funder of the CGS? According to written and recorded statements from current and former DARPA program managers, as well as other government documents, the DARPA-funded educational video games developed at the CGS have a purpose beyond the pretense of teaching elementary school children STEM skills.
Instead, the games developed at CGS have had the primary purpose of using grade-school children as test subjects to develop and improve "adaptive learning" training technology for the military.
It's one node in a network of programs originally created with the intent to build better counterinsurgency simulations for American warfighters operating in countries occupied by US forces. In short, DARPA is militarizing academia. But it's not that easy.
There isn't much to the place. Seth Cooper, CGS's creative director, gives me the quick tour. There's a small computer lab and an adjacent office, all blanketed in cheap carpets and fluorescent lights. A deflated Stars and Stripes Uncle Sam hat rests on one of the tables.
In a small lunch room, Cooper and I discuss CGS's most famous video game, Foldit, a protein-folding puzzler that crowdsourced research previously only done by experts, making "citizen researchers" out of every player. One puzzle in the game involved a mystery surrounding the cause of the AIDS virus in rhesus monkeys that had stymied scientists for 15 years. Foldit players solved the mystery in ten days.
The lab's newest "scientific discovery game" is NanoCrafter, which it hopes will have even greater success in democratizing scientific research. The CGS has enjoyed some press of late, specifically about NanoCrafter, to the point that everyone I spoke with who was involved with the lab seemed to think that was my purpose as well. I was really there to see about their relationship with the military.
The CGS website cites DARPA right on its main page, as well as the Bill & Melinda Gates Foundation, the Office of Naval Research, and several corporate sponsors. But my curiosity was piqued by a 2011 document from the University of Washington declaring CGS's founding with $15 million dollars—the total of a cool $3 million from the Gates Foundation, and $12 million from DARPA.
The Gates Foundation's involvement isn't all too surprising. The foundation is committed to making the world a better place, after all. Its worldwide headquarters are also in Seattle and Bill Gates happens to be a local geek. Gates mentioned the Center, as well as director Zoran Popović, in the 2012 Gates Foundation's Annual Letter, and again during remarks at the 2012 Education Commission of the States Annual Conference.
Afte Foldit, CGS had begun creating educational video games that utilized adaptive tutoring technology, which taught kids math and other STEM skills as they played. Gates was clearly fascinated, shouting out Popović in the 2012 Annual Letter for spearheading the creation of:
...games that automatically adapt to each student's unique needs, based on their interactions with the computer. Many of these new tools and services have the added benefit of providing amazing visibility into how each individual student is progressing and generating lots of useful data that teachers can use to improve their own effectiveness.
Gates took that sentiment a step further in his speech at the Education Commission Conference, when he asked the audience to imagine a world where the underlying mechanics of the video games kids willingly poured themselves into imparted mathematics lessons "while they barely noticed because it was so enjoyable." To that end, he explained, "We've been supporting the Center for Game Science at the University of Washington."
Repeated requests for comment from the Gates Foundation have gone unanswered. I'll be sure to update this story if I hear back.
Despite all this, the Gates Foundations' monetary contribution was dwarfed four-fold by DARPA (whose new 'Squad X' research initiative, in case you missed it, looks a lot like Call of Duty). What was DARPA's interest, and how does this lopsided foundational grant influence the CGS? As a recipient of federal dollars harvested from taxpayers, it's worth considering that there is, in fact, a Department of Education. So why was this project being funded by the military?
Kids barely noticed they were learning because the game was so enjoyable.
Back in the kitchen, I ask Cooper how receiving this sort of funding from both The Gates Foundation and DARPA worked. Did it all come no strings attached? Or was there some direction?
"Oh, there is direction," Cooper told me. "It kind of depends on the specific game, or that kind of thing. There are particular goals and objectives." These goals and objectives, he adds, come directly from DARPA or the Gates Foundation.
"You write a proposal that you are going to do particular things and then that gets funded, and then you are supposed to do those," explained Cooper.
Fair enough. But what's in it for DARPA, such that it would allot $12 million to kickstart the CGS?
For "those sorts of things," Cooper said, I'd be better off speaking with Popović himself.
First, I should say that we know the Center for Game Science is being funded through a DARPA program called ENGAGE (though they're often capitalized, I'm told the letters don't actually stand for anything), and that it was Daniel Kaufman, current head of the Information Innovation Office (I2O) within DARPA, who had the first "seedling" idea for what became ENGAGE.
Perhaps coincidentally, Kaufman had previously been co-chief operating officer at Dreamworks Interactive, the Steven Spielberg-founded game studio that created the commercial video game series Medal of Honor, a first-person shooter military game that has reportedly been used for training purposes in the Marine's Infantry Cognitive Skills Labs.
But it was after he learned of the work that Zoran Popović and others were doing with Foldit that it dawned on Kaufman. Today, DARPA's I2O self-describes the mission focus as "cyber and other types of irregular warfare," with a research portfolio focused on forecasting "new modes of warfare in these emerging areas and developing the concepts and tools necessary to provide decisive advantage for the US and its allies."
In response to a solicitation from Kaufman, a program manager at the time, and an initial $500,000 grant, Popović submitted a white paper outlining the untapped potential of video games. His proof of concept was the crowdsourced research game Foldit, but Popović also demonstrated that adaptive tutoring technologies could be created from what they had learned on that project. At which point another program manager, Captain Russell Shilling, developed these ideas fully into what is now the ENGAGE program, triggering all those millions of dollars in funding to found the CGS.
Captain Russell Shilling has since left DARPA; program managers with the Pentagon's blue-sky research arm typically only stay on between four and six years. In June 2014, President Obama appointed Shilling as the Executive Director for STEM Initiatives at the Department of Education.
I managed to get a message to Shilling—whose Twitter headshot, if you're curious, is a Muppet of himself—asking why it was DARPA, not the Department of Education, that was funding a project based on elementary STEM education.
"I can't really speak for DARPA, since I'm no longer assigned there," Shilling wrote back. Instead, he pointed me to two newsletters, both of which he authored, available on DARPA's website that he said "explain the intentions." One is about some experiments with game-based training, and the other is about the ENGAGE program specifically.
According to this second newsletter, titled ENGAGE: New Methods of Large Population Analytics for Education and Training, teaching math and science was not the primary goal of the ENGAGE program. Rather, the intention was to use the video games that the CGS had developed in conjunction with DARPA—games like Treefrog Treasure and Refraction—to experiment in teaching elementary school students STEM skills as a way to test the games' adaptive tutoring capabilities with thousands of subjects.
Once perfected in this manner, Shilling goes on, the technology behind adaptive tutoring could be applied to other fields of training more directly fit for military use. There would be no shortage of test subjects either, because, as the newsletter notes, DARPA has access to the children of military personnel around the world through the Department of Defense Education Activity, a civilian wing of the Pentagon through which this software can be tested.
The DARPA newsletter also takes credit for the Algebra Challenges that the CGS promotes, noting that thousands of kids took part in them. Mind you, the actual Algebra Challenges website mentions neither DARPA sponsorship, that the kids involved are taking part in military testing, nor that the DoD is collecting all that data.
That said, Shilling writes about how ENGAGE harnesses the power of big user pools "to optimize instruction." Per the newsletter:
Where traditional education tested effectiveness in small samples (<100) the ENGAGE approach to assessment involves thousands or tens of thousands of students… ENGAGE has initially focused on interactive technologies for K-12 students, including the Department of Defense Education Activity, a government-sponsored organization responsible for educating the children of military personnel around the world.
Shilling's newsletter concludes by saying we can anticipate that the same tactics the ENGAGE program utilized in optimizing instruction in math and sciences " can be applied to a wide variety of military… training contexts." (Emphasis mine.)
What is this wide variety of training contexts? At first I thought, wrongly, that the STEM education for kids would simply be adapted to STEM education for soldiers. But it's intended to be applied to a "wide variety" of military training contexts, not merely education. So the question becomes, what specific training ideas, if any, did DARPA or the DoD have in mind for this adaptive tutoring technology?
DARPA's (unclassified) 2012 budget offers a clue. Written in February 2011, the 2012 budget marked the first appearance of ENGAGE, and elaborates at length on the program's roadmap for the years 2011-2012. At no point anywhere does it mention STEM education, let alone STEM education for kids, being a goal.
Instead, its stated aim is to seek ways to improve adaptive teaching AI for game-based training that could make the leap to the real world. "The ENGAGE program, previously part of the Training for Adaptability thrust," the 2012 budget reads, "...will address the problem of connecting performance in the virtual domain and then will use this knowledge to drive the creation of more effective game based training." ENGAGE will also "develop approaches for extrapolating performance on computer-based training systems to performance in the real world."
Previously part of the Training for Adaptability thrust? What is Training for Adaptability?
If you go back to DARPA's 2011 budget, Training for Adaptability is loosely identified, without much detail, as a leadership training program. This makes it hard to see how it might track with ENGAGE. But buried in the 2012 budget is something called Strategic Social Interaction Modules (SSIM), with a note that reads, "Formerly Training for Adaptability."
This line item is key. It's talking about a training system for US service members in "Counter-Insurgency (COIN) missions," the goal being to teach warfighters how to successfully deal with foreign civilians living under US occupation while "ultimately winning their hearts and minds." Per the 2012 budget line item:
SSIM will develop requisite training technology including advanced gaming/simulation techniques…develop the tools to identify skillful performance in a training environment and for predicting the efficacy of the training in the intended operational/cultural environment.
For comparison, here again is Bill Gates' description of the technology being developed at the CGS: "These new tools and services have the added benefit of providing amazing visibility into how each individual student is progressing and generating lots of useful data that teachers can use to improve their own effectiveness."
In the 2011 budget, the year before the ENGAGE program was listed, DARPA had another preexisting program going that seems to provide more even insight into the background thinking of ENGAGE. This program, called "Training Superiority," is part of the DARWARS initiative, a plan to improve training through simulations and video games. The 2011 budget speaks of DARPA's ambitions to create an intelligent tutoring system called Digital Tutor.
"The Training Superiority program," the line item description reads, "will change the paradigm for military training." Traditional, passive teaching approaches are no longer sufficient in the modern battlefield, it goes on, as more is demanded of fewer soldiers, who must become proficient in controlling and interacting with complex unmanned systems.
Modern training thus must include computer games' emotional element, the thrust of which will scale-up new digital tutor methodologies, and "deliver these to a large cohort of warfighters."
But then there's this government document that lays out training superiority and the Digital Tutor in detail, bizarrely ending with—and this is not a joke—a line from Machiavelli about "a new order."
The current program manager of ENGAGE, the man who took over from Russell Shilling, is Daniel Ragsdale, a recently retired US Army Colonel whose combat deployments include, his DARPA bio reads, "Enduring Freedom" and "Iraqi Freedom," those oft-forgotten official designations of the post-9/11 wars in Afghanistan and Iraq. Academically, Ragsdale's research interests include cyber deception.
When I asked Ragsdale in a phone interview why he's funding the Center for Game Science, he explained how he had the good fortune of inheriting the program from Shilling, who he said created the program, first and foremost, with an eye to STEM education.
So the point of ENGAGE, and the impetus behind its funding, Ragsdale explained, is thus to develop both game-based technologies that accelerate learning and "new approaches" for analyzing gameplay, all while adapting to individual and large groups of people.
"This is the real important part," he stressed, "so it's not just games for the sake of games or educational games, but specifically games that will adapt over time to specific individuals or groups of folks." The target being not only classroom settings, but also for individual use for STEM, Ragsdale said, adding that for STEM targets, "basically we were looking at [grades] K through 3, K through 4, that demographic."
As for why DARPA was even pursuing STEM educational video games for elementary school kids, Ragsdale gave a somewhat surprisingly offhand confirmation of what I had found in the budgets and other documents. "The real inspiration for the program was not STEM," he said. "We wanted to develop new methods that are game-based to accelerate learning and have something that adapts."
With that stated goal, Ragsdale added, the question then was, What's a good target? And as he told me, "It was not 'let's do a STEM program.'"
So what you're learning with these kids is scaleable to a whole range of things if you have an intelligent tutoring system? I asked.
"Absolutely, that's our real thrust," Ragsdale explained.
The real inspiration for the program was not STEM Education.
It reminded me of another game, called Ambush, part of the DARWARS Training and Superiority Initiative, which trained soldiers to anticipate and react to the threat of IEDs and ambushes in Iraq and Afghanistan. Could what's being learned here with adaptive tutoring be applied to military simulations the likes of Ambush?
"Yes, yes, I think that's fair to say," Ragsdale said.
It's not the children's educational games being developed at the Center For Games Science that is interesting to the military—it's the intelligent tutoring technology. The kids are just unwittingly testing it out for them.
I finally caught up with Zoran Popović, who had been out of the office when I'd visited the CGS, and asked him about the $12 million from DARPA.
He told me the money was spread over four years, with another year-and-a-half or two years of support still forthcoming. But he had heard rumors that certain senators had gotten wind of DARPA's involvement with the CGS, and were unhappy with the military's involvement with children's educational games when there were already other funding agencies, such as the Department of Education and the National Science Foundation, under whose purview such a thing might more aptly fit.
"There was, at least from what I understand, some pushback," Popović said.
But he did mention that the program would continue with funding from other agencies. For example, over the last four years the National Science Foundation has given the GSC roughly $3 million, according to Popović. He estimated that the money they'd received from the Gates Foundation has only been around $1.5 million to $2 million.
Popović also mentioned in a follow up email that the CGS funding from DARPA is much smaller than that of SRI, the research center formerly known as Stanford Research Institute. In 1970, SRI was pressured by student protesters to separate from Stanford due to their DARPA funding, which was seen during the Vietnam War as an unwelcome intrusion of the military-industrial complex into a place of learning.
"But I hope you don't write too much about the input," Popović told me. "The output is what matters, right? I'm not interested in raising a lot of funds. I'm interested in doing transformational things."
I believe him, for what it's worth. It seems the Center for Game Science is just taking funding from where it can get it. And from the military's ENGAGE program, has come the Gates Foundation-funded ENLEARN, which uses the same technology being developed by the CGS to teach kids through video games, but evidently without any ulterior motives.
I asked Henry Giroux, a cultural critic and author of The University in Chains: Challenging the Military-Industrial-Academic Complex, what he thought of military funding for projects that also had social benefits.
"I am not averse to universities receiving money from the military, Defense Department, or any other government agency, so long as there are no strings attached as to how that money should be used," Giroux said in an emailed statement. "Once that principle is violated, the door is open for more anti-democratic values and projects to move into the space of the university."
DARPA was created by President Eisenhower, beneath a Soviet-launched satellite, Sputnik, the first and only satellite then, which stared down at us from violated American skies. Ike was also the man who, in his 1961 farewell address, simultaneously recognized the need for a strong military but also warned of its dire side-effect: the capacity to unmake a free-thinking republic.
As research has become "central," Ike argued, so too has it become more "formalized, complex, and costly":
A steadily increasing share is conducted for, by, or at the direction of, the Federal government… [T]he free university, historically the fountainhead of free ideas and scientific discovery, has experienced a revolution in the conduct of research. Partly because of the huge costs involved, a government contract becomes virtually a substitute for intellectual curiosity. The prospect of domination of the nation's scholars by Federal employment, project allocations, and the power of money is ever present – and is gravely to be regarded.
Which brings us back to Foldit and NanoCrafter. Both games are bringing scientific research to the everyman, as Ragsdale had told me, his sincerity palpable over the phone from Arlington, Virginia. "Nanocrafter, I believe, is going to be disruptive technology for science in general," Ragsdale said. "It could revolutionize science and biochemistry, and I don't think that's an overstatement."
It's interesting that the CGS first got the attention of DARPA after developing Foldit, the crowdsourcing research game, but wound up making adaptive educational games for the futuristic research arm instead, which is not the same thing. Is the CGS's ability to crowdsource solutions to science problems, then, not being put to use by DARPA?
I have found little grumbling from researchers about getting Defense Department funding.
In fact, DARPA, originally as part of a separate program than ENGAGE but now also handled by Ragsdale, sought proposals to crowdsource the testing and debugging of software, so that it could be done inexpensively by nonexperts. The Center for Game Science submitted the best pitch, winning the contract and funding for a project called Crowd-Sourced Formal Verification.
In the end, the CGS solution was to create video games with visuals representing the underlying mathematics, so that as you played you tested the software for defects. Verigames, a process for detecting software flaws, was born.
Verigames looks like another portal for kids games, and it doesn't explain on the site what sort of software you are testing. But as DARPA's original Crowd-Sourced Formal Verification Proposer's Day Announcement makes clear, as you play these games you are taking part in debugging military weapons systems software.
Since I had been reading his book, The Department of Mad Scientists, for background on DARPA, I reached out to author Michael Belfiore, asking if researchers ever had ethical issues with receiving military funding.
"I have found little grumbling from researchers about getting DOD funding and the possible ethical concerns there," Belfiore told me. "Most seem to agree that beneficial research is beneficial research, whatever the source of funding."
The games at Sieg Hall, for all we know, have only just begun.