Apparently the Web's "Smartest" Search Engine Is Dumber than a Six-Year-Old

Search engines can solve math equations in seconds, serve up volumes of info on nearly any subject, and translate text into hundreds of different languages. Having indexed the world's knowledge and all, Google comes off like a big smartypants. But how intelligent is it, really?

To answer this question, researchers at Beijing Jiaotong University made some of the web's most popular search engines, including Google, Bing, and Baidu and So in China, take an IQ test with a set of questions designed to gauge their knowledge. Then, they gave the same test to a bunch of kids aged 6 to 18 and compared the results. As it turns out, the internet's "smartest" search engine, So, is only about half as smart as a six-year-old.

The experimental study, published in Procedia Computer Science, set out to build a metric by which we can measure the internet's intelligence as it develops. Recently, major search players like Google and Baidu have begun developing deep learning neural networks—complex computer algorithms that "teach" themselves to recognize meaning and concepts—to parse user-submitted images and queries.

These advancements in machine intelligence have some people, like Nick Bostrom and his colleagues at the Future of Humanity Institute, worried. Others, like Selmer Bringsjord and David Ferrucci—the guy who developed Watson for IBM—aren't so sure that we're headed towards a Terminator-esque robot apocalypse any time soon, if ever. For the researchers behind the study in question, these breakthroughs are an invitation to see how search engines are stacking up to human intelligence.

Inspired by the human version of IQ testing, which runs participants through a gauntlet of questions meant to test their knowledge and cognitive skills, the researchers designed a base set of questions for search engines. To test for common knowledge, for example, they asked for the name of the longest river in the world. To gauge creative knowledge, the search engines were asked to associate concepts—birds are to the sky, as fish are to what?—and tell a story with a given set of words.

The test isn't at all perfect. Parts contained assumptions that disadvantaged some search engines compared to others; in one instance, So was the only engine to score on the sound input test because it was the only one with the ability to detect sound. Hardly fair. Creative questions were also weighted more heavily than ones that dealt with knowledge retrieval, so it's not too surprising that none of the machines tested much higher than a grade school intelligence level.

In the end, none of the search engines outperformed any of the kids who took the test. So, the "smartest" of the bunch, according to the test, only managed to attain an IQ score of 25. The average score for the 6-year-old test bracket was 55.5. Though the search engines beat the kids in knowledge retrieval categories, they bombed the creative ones.

At the end of the day, it's a silly little experiment. But it's yet another reminder that artificial intelligence that's actually par with that of humans—which includes the ability to be creative—is likely a long way away.