Canada Is Experimenting on Intelligence Analysts to Beat ‘Information Overload’
And it could help you, too.
Image: Flickr/Marcel Oosterwijk
On the battlefield, shoddy intelligence means innocent people die. To make its intelligence analysts more effective, the Canadian government is experimenting on them by treating these highly trained personnel like animals.
No, really, intelligence analysts aren't so different from wild animals, and that's actually the point. Like hungry little foxes, they go scrounging around different nooks and crannies, except they're hunting in various databases for satellite images, not a tasty critter to eat. This is called "information foraging," a theory that originated at the storied Palo Alto Research Center, where much of modern computing was born, in the early 1990s.
The Canadian military wants its intelligence analysts to get better at this kind of foraging because, according to Defence Research and Development Canada (DRDC) researchers, analysts face two major challenges: "information overload" and tight time constraints. This makes sense when you consider how Canada and other governments have accelerated and expanded their digital surveillance regimes over the years—that's a lot of data.
To this end, DRDC launched a project in 2014 involving a whole constellation of research on how to make intelligence analysts do their jobs better. Part of that project is INFOCAT, an experimental platform for testing the information foraging abilities of analysts, which can then be used to design better training and search systems.
"In information foraging, you might Google something and as you go along, you'll find items that aren't particularly useful or helpful. It's a question of diminishing returns"
The work with INFOCAT is being led by cognitive psychologist David Bryant in Toronto, and the team just launched its first experiment, he told me when I interviewed him over the phone. A DRDC spokesperson was also on the line.
"Animals ask themselves: While I'm foraging in this bush, how long should I stay there?" said Bryant. "Should I stay there and completely exhaust the food? That's not a very good strategy."
It's a simple question of economics—how long do you dick around in one bush, er, database looking for the last few berries before you move to another, more fruitful location, and how fast can you do it? To measure this in people, INFOCAT gives subjects just 20 minutes to answer a research question with lots of databases at their disposal.
"In information foraging, you might Google something and as you go along, you'll find items that aren't particularly useful or helpful," Bryant explained. In other words, how long do you plod through pages of increasingly unhelpful search results before you try a different combination of words? "It's a question of diminishing returns," Bryant said.
This work will have applications outside of a military context, Bryant said. After all, in 2016, information overload isn't just a problem for military types. As a steady stream of individual tweets, news stories, memes, and occasionally relevant information inundates us all, sussing out what's important and what's not can be tough. A system that helps you navigate it efficiently would be helpful for academics or businesspeople, or, hell, the average Twitter user.
Watch more from Motherboard: America's Ex-Drone Pilot
Canadian intelligence analysts prepare briefings on anything from strategic questions, like whether ISIS is likely to attack a certain region, to tactical ones, like which frequencies a particular radar emits, Bryant said. Analysts must look through a ton of different information sources—signals intelligence such as metadata, human intelligence reports (classic, "boots on the ground"-style spook stuff), satellite imagery, and books or Wikipedia.
The job is to pull out relevant information, without spending too much time in any one source. You know, like an animal hungrily scurrying from place to place looking for a bite to eat.
"In the experiment, we vary how the information is distributed in databases, we vary the cost associated with moving from one database to another, and the cost of opening an item and processing it," said Bryant. "By varying these factors, we can create situations where we'd expect different behaviours from an optimal forager. We can see if their behaviours are in line with what information foraging theory predicts."
The nerdier among us may be asking ourselves: instead of going through all this trouble, why not just automate the whole process? According to Bryant, there's no AI right now that can reliably do the cognitively complex job of an intelligence analyst, although that may one day be the case. For now, we have many people who do this job, and so Bryant and the DRDC are focused on making them better, instead of replacing them.
That doesn't mean that machines don't have a part to play in that mission, however—a 2014 DRDC research paper saw the military looking into creating an AI-based virtual assistant to assist intelligence analysts.
In the information-drenched future of war, human analysts—animals that they are—are going to need a little help.