Now that more and more data is encrypted, it will be harder and harder for governments to spy—unless internet companies help them.
Last year, the US government asked Yahoo to scan all its customers emails to look for the digital "signature" of certain method of communication used by a terrorist group.
The secret scanning tool rocked the tech world and that of privacy and anti-surveillance activists this week, shocked by how broad the request was, and the fact that it had kept under wraps. Until Reuters reported its existence this week, the tool remained a secret not just to the public, but also to most people at Yahoo, including the security team, which caught it and thought it was a sophisticated and dangerous piece of malware installed by hackers, as reported by Motherboard on Friday.
In an age where the FBI asked Apple to unlock the iPhone of a dead terrorist, and countless top secret documents revealed the vast surveillance powers of the NSA, asking Yahoo to use a scanning tool that former employees defined as a "buggy" and a "poorly designed" "backdoor" or "rootkit," was an unusual request as it wasn't targeted—Yahoo was essentially looking through the whole haystack looking for a few needles—and the company apparently didn't fight back.
But such a request is a perfect example of how the rise of encryption technologies have changed the face of government surveillance. Before large tech companies such as Google and Yahoo turned on default encryption across their services, including email—in the process protecting their customers data as it travelled from their computers to the company's servers, as well as when the data travelled through the internet—the NSA could've gone through users' data without having to knock on Yahoo's door.
"Five years ago the government would've just done this by spying on international telecommunications traffic."
"Five years ago the government would've just done this by spying on international telecommunications traffic," Christopher Soghoian, the principal technologist at the American Civil Liberties Union, told Motherboard.
As revealed by documents leaked by Edward Snowden, the NSA and its British partner GCHQ have taps on the internet backbone infrastructure, and used to collect data directly from what they called "upstream" provides, none other than large ISPs. Before the Snowden revelations, when companies such as Microsoft, Yahoo, and other email providers didn't use encryption, all these emails were travelling in the clear over the internet, and were collected by spy agencies.
Especially in the aftermath of the Snowden leaks, companies started adopting HTTPS, which encrypts connections between users and the server, and STARTTLS, which encrypts emails as they travel through the internet between providers. In 2014, Google reported that, for the prior year, only 33 percent of emails between Gmail and other providers were encrypted. The number rose to 58 percent in 2014, when major providers like Yahoo and Microsoft also adopted STARTTLS. Now, it's almost pervasive, at more than 80 percent.
"Every time a provider turns on STARTTLS the NSA loses access to more and more email," Soghoian told me. "NSA has lost a good percentage of its visibility into international email traffic without the help of tech companies."
Without the ability to just ingest emails travelling across the internet and looking through their content, the NSA has to ask the Yahoos, Googles and Microsofts of the world to do that for them now. In a way, this is the same issue the FBI and police are encountering when trying to break into iPhones, or intercepting end-to-end encrypted messaging apps like Signal or WhatsApp. The data is now encrypted, and without some help hacking in, there's no way in.
That's just how surveillance works today, and there's no turning back.
Get six of our favorite Motherboard stories every day by signing up for our newsletter.