Image via Flickr
Just when you thought things over at the Department of Defense couldn't get any nuttier, the Pentagon is now crowdsourcing US national security—to protect the country from the looming threat of too much data.
DARPA just announced it's investigating the national security threat posed by the large sets of public data culled from websites and social media. It's calling for proposals from data scientists to develop tools to make sure the mass of readily available personal data doesn't fall into the wrong hands.
"Current trends in social media and commerce, with voluntary disclosure of personal information, create other potential vulnerabilities for individuals participating heavily in the digital world, DARPA writes. The agency will accept proposals from August 26 to September 25.
If you just got a big ol' whiff of irony, you’re not alone. While the right hand of the DoD is collecting loads of personal data on Americans in the name of protecting national security, the left is worried the personal information being tracked and stored is a national security threat. Can we get on the same page here?
Granted, the phone records and online communication being surveilled by NSA to thwart terrorism isn't exactly the same as the massive amount of personal information collected and analzyed by data aggregators—companies like Acxiom or ChoicePoint that are trolling every site we visit, every status update we send, every video we watch, to create an online profile of who we are and what we like, and what we're likely to buy. This is the stuff DARPA is worried about.
For example, this investigation was prompted by the 2009 Netflix Challenge to perfect the website’s algorithm—now famous for accurately predicting what users want to watch based on big data. In the process of reverse-engineering the site's algorithm and making that information publicly available, programmers in the contest inadvertently outed a closeted lesbian who had watched Brokeback Mountain. It led to a federal lawsuit over privacy invasion, and raised the question: What other secrets can be gleaned?
The thinking is, what's to stop the "bad guys" from doing essentially the same thing, analyzing personal information that can be used in nefarious ways? Say, spying on military plans, waging a cyberwar, gleaning national security secrets? DARPA writes:
Does the availability of data provide a determined adversary with the tools necessary to inflict nation-state level damage? To what extent could a non-state actor collect, process, and analyze a portfolio of purchased and open source data to reconstruct an organizational profile, fiscal vulnerabilities, location of physical assets, workforce pattern-of-life, and other information, in order to construct a deliberate attack on a specific capability?
The Pentagon is up to its ears in data—more than it knows what to do with. Even with our best tools, information is being collected faster than we can figure out how to analyze it. This February, a defense official said data scientists were the most in-demand job in the military, which is competing with private companies to try and snatch up these experts.
Now, not only is the Defense Department struggling with how to extract value from big data, it's making sure the whole online intelligence trend isn't going to horribly backfire.