Six Ways Law Enforcement Monitors the Dark Web

​Law enforcement and governments are increasingly trying to crack the anonymizing network Tor and the dark web crime rings that operate there. Now, a research group is suggesting that dark web monitoring should become a primary goal of internet governance bodies.

The Internet Corporation for Assigned Names and Numbers' contract with the United Nations is set to expire​ this year, which means it's a good time for internet governance types to try to make changes to how it operates. Some believe ICANN should become less US-centric. Others believe ​ICANN needs to focus more on understanding the dark web.

Michael Chertoff, former secretary of the Department of Homeland Security and now a security advisor and lawyer, and Tobby Simon, of the Bangalore-based Synergia Foundation think tank, just ​released a paper with the Global Commission on Internet Governance suggesting that researchers need "new ways to spot upcoming malicious [dark net] services to deal with new phenomena as quickly as possible."

Chertoff actually compares fighting crime on the dark web to Alexander the Great's doomed invasion of Persepolis in 331 BC. The "Battle of the Persian Gate," in which Alexander's army was ambushed by unknown forces, should "remind us of the importance of reconnaissance, and the need to better understand what is beneath the surface."

It's a dramatic comparison, but not all that surprising coming from the former head of what has largely become a law enforcement agency. Chertoff does acknowledge some of the above-board uses of the dark web, notably for whistleblowers and oppressed peoples, but he also details every method we know of to scan the dark web.

Mapping the hidden services directory

This is most likely what the ​Justice Department meant earlier this month when it said it has "made some advances in [its] ability to penetrate the Tor network." Tor operates on a series of distributed "nodes," which are run by volunteers all around the world. Tor traffic routes between these nodes, anonymizing data. But if you own and operate enough nodes you can glean enough information to learn quite a lot about the dark web and about who uses it. A recent report from the Tor Project suggests that 30,000 hidden service websites make up less than 1 percent of all traffic routed through the network.

Customer data monitoring

The NSA and other intelligence agencies could "benefit from analyzing customer web data to look for connections to non-standard domains," Chertoff wrote. In other words, he's suggesting that if security agencies can tell people are going somewhere on the dark web, they can make inferences from it. "This can be done without intruding on users' privacy as only the destinations of the web requests need to be monitored and not who is connecting to them" he wrote, which seems like wishful thinking.

Social site monitoring

Regular web sites such as Pastebin, where links to dark web sites are often posted, should be "kept under constant observation," Chertoff wrote.

Hidden service monitoring

Dark web sites go offline and resurface all the time—Chertoff notes that it's "essential to get a snapshot of every new site as soon as it is spotted, for later analysis or to monitor its online activity."

Semantic analysis

Once a dark web site is found, it should be downloaded and its information should be put into a database for future analysis and to compare to other hidden services to "associate them with malicious actors," he suggested.

Marketplace profiling

He suggests that security agencies should create databases of dark web dealers—the same dealers often move between different drug markets, for instance. "Individual profiles could be built up over time," he wrote.

The dark web is still something of a wild west, but security agencies have increasingly spent lots of time trying to bring down sites like Silk Road. That doesn't sound likely to change anytime soon, and it seems like international governments may soon take a hard look at doing whatever they can to map everything that lurks just below the surface.