The Cybersecurity World Is Debating WTF Is Going on With Bloomberg’s Chinese Microchip Stories

On Tuesday, Bloomberg doubled down on its bombshell report from last week, which alleged China had surreptitiously implanted tiny chips into the motherboards of servers to spy on US companies such as Apple and Amazon. If true, this would be one of the worst hacks in history.

In its new story, Bloomberg reports that a US telecom discovered and removed “manipulated hardware” in its servers. The article does not name the telecom and the key claims are all attributed to one source, Yossi Appleboum, co-CEO of security consultant Sepio Systems. Bloomberg reports Appleboum provided “documents, analysis, and other evidence,” but does not publish those or provide more information about what types of documents or evidence it has.

It is not clear in the article that Bloomberg knows which telecom is apparently affected; it notes that Appleboum is covered by an non-disclosure agreement. Motherboard has reached out to 10 major US telecom providers, and the four biggest telecoms in the US have denied to Motherboard that they were attacked: In an email, T-Mobile denied being the one mentioned in the Bloomberg story. Sprint said in an email that the company does not use SuperMicro equipment, and an AT&T spokesperson said in an email that "these devices are not a part of our network, and we are not affected." A Verizon spokesperson said: "Verizon's network is not affected.”

A CenturyLink spokesperson also denied that the company is the subject of Bloomberg's new story. A Cox Communications spokesperson said in an email: "The telecom company referenced in the story is NOT us." Comcast also said it's not the company in the Bloomberg story.

On Monday, Apple also doubled down, with a new strong denial sent to multiple Congressional committees. The company sent a letter refuting the first story, published in Bloomberg's Businessweek, which said China had planted hardware backdoors onto motherboards made by a company called SuperMicro used by multiple US companies, including Apple and Amazon.

The letter is the strongest signal yet from a growing array of government agencies, companies, and technical experts who are calling the Bloomberg story into doubt. (The new story does not directly address these denials.)

“You should know that Bloomberg provided us with no evidence to substantiate their claims and our internal investigations concluded their claims were simply wrong,” the letter, signed by George Stathakopoulos, vice president of information security at Apple, reads.

“Apple has never found malicious chips, ‘hardware manipulations’ or vulnerabilities purposefully planted in any server. We never alerted the FBI to any security concerns like those described in the article, nor has the FBI ever contacted us about such an investigation,” the letter continues.

Bloomberg's blockbuster piece published last week promised to be our worst fears about supply chain attacks realized. The article claimed the Chinese government had managed to plant tiny, extra chips onto the motherboards of SuperMicro, a computer parts supplier that produces server parts for Apple, Amazon, and many others. Those chips could have given Chinese hackers privileged access to those companies’ systems, the piece said.

“Hardware is a nightmare. We can barely validate software, and all our assumptions rely on the hardware working correctly. Pull away that assumption, it’s like removing the screws from a piece of IKEA furniture,” Matthew Green, associate professor at Johns Hopkins University, previously told Motherboard in an online chat. The article itself was based mostly on anonymous sources, both inside impacted companies and those who had been briefed on the incident.

The fallout, or rather backlash, against the Businessweek piece has been dramatic.

First, Apple, Amazon, and SuperMicro all issued largely unambiguous statements pushing against the story; it is a relatively unusual move to have such robust, refuting statements that provide little wiggle room for a story’s claims to hold up. Next, the UK’s National Cyber Security Centre (NCSC), the defensive arm of the country’s signals intelligence agency GCHQ, issued its own statement saying it had no reason to doubt Apple’s and Amazon’s denials. The US Department of Homeland Security (DHS) swiftly followed up, saying much the same thing. Apple’s recently retired general counsel even called his FBI equivalent last year after being told by Bloomberg of an investigation into SuperMicro: “Nobody here knows what this story is about,” James Baker, the FBI’s then-general counsel, said, according to a report from Reuters.

In a statement to Motherboard sent on Monday, a Bloomberg spokesperson reiterated that it stands by the story, “and are confident in our reporting and sources.” (Bloomberg did not immediately respond to a request for comment Tuesday after the new story was published. One of the co-authors did not respond to a Twitter direct message. )

An Apple employee with direct knowledge of the company’s internal investigations who spoke on condition of anonymity told Motherboard that “none of the most consequential portions” of the original Bloomberg story as they relate to Apple are true. They said the company did not find malicious chips in its servers, it did not remove or dispose of those servers, and added that Apple did not inform the FBI nor frustrate an investigation into this incident .

“In early 2016, our Information Security department identified an infected Windows server in one of our labs. An investigation revealed the infection to be the result of malware-infected software drivers downloaded from the Super Micro site,” the Apple employee told Motherboard. “No infected firmware was found. The infection of the drivers was determined to be accidental—the result of poor system hygiene by the vendor—not a targeted attack against Apple. Following our discussions with them, we severed the relationship last summer.”

The employee said Apple launched an internal investigation to look into the allegations Bloomberg brought to the company.

“It was cross-disciplined, detailed and rigorous,” the source said of the investigation, explaining that it involved reviewing purchase orders, finance documents, logs, security records, and interviews with employees who would have known or been involved in such an incident. No Apple employee “had ever heard of anything even loosely matching what Bloomberg was describing,” the employee said.

Because the allegations made in the Bloomberg story would be world-shifting, the story has dominated the discourse in the cybersecurity world. Though proactively detecting a malicious chip in the supply chain is very difficult and time consuming, security experts say that detecting one after the fact should be trivially easy.

“I think a lot of information is missing,” Omer Shvartz, a security researcher at Israel’s Ben Gurion University who showed a proof-of-concept supply chain hack involving a malicious chip in his research last year, told Motherboard in an email. Shvartz said the Bloomberg story is certainly plausible but that he has lots of questions.

“I would like to see a decapped chip (the exposed silicone structure after dissolving or scraping the plastic envelope) or at least some sort of radiographic image of the contents,” Shvartz told Motherboard. “This is standard procedure for analysis of suspected electronics and will allow us to understand what are the capabilities of the device.”

Bill Cardoso, the CEO of Creative Electron, a company that sells x-ray machines that are designed, in part, to detect malicious chips in electronics, told Motherboard that it has developed technology that can identify this sort of hack pretty easily after the fact.

As early as 2016 “we were doing work with a commercial organization to provide them with an x-ray system to inspect boards coming from different vendors,” Cardoso said. “The x-ray system we sold them was powered with an artificial intelligence engine that learned what a good board look like, and over time was able to pick up even the smaller of foreign objects.

But, as Zack Whittaker wrote at TechCrunch, reporting on national security issues using largely anonymous sources makes it difficult for Bloomberg to prove to its readers how it knows what it knows, which has led to widespread doubts about the original article’s veracity when faced with such strong denials from nearly every company involved and the intelligence community. Because of the sensitive nature of the reporting, we don’t have a chip, or photos of it, and Bloomberg wasn’t able to publish any of the source documents that it said it has seen.

On Monday night, former White House cybersecurity czar, and now NSA advisor Rob Joyce added his own grain of salt soliciting more information about the story on Twitter: “Still interested if someone is directly connected to a discovery of adulterated hardware-DM me”—implying that he’s not aware of a China-led supply chain attack such as the one reported by Businessweek.

Even sources used in the original story are confused about what’s going on. The cybersecurity podcast Risky Business interviewed one of the few named sources in the original Businessweek article, hardware security expert Joe Fitzpatrick, who expressed doubts about the article, and said he had never been contacted by any Bloomberg fact-checker. Fitzpatrick was used as an expert source to comment on the technical details of what Bloomberg described and does not have any firsthand knowledge of the actual alleged hack.

“I have the expertise to look at the technical details and I have the knowledge to look at the technical details and see that they’re jumbled. They’re not outright wrong, but they are theoretical.” Fitzpatrick, a well known hardware security trainer, told Risky Business founder and host Patrick Gray. “I see a lot of details that I gave out of context, so I’m not an expert judge on quality of journalism, but I definitely have my doubts on this one.”

The prospect of this kind of attack is very real, but the fact that both Bloomberg and the companies named in the story are doubling down is confusing everyone, and a sign that we are probably not done hearing about this story anytime soon.

A Bloomberg spokesperson said in a statement: "As is typical journalistic practice, we reached out to many people who are subject matter experts to help us understand and describe technical aspects of the attack. The specific ways the implant worked were described, confirmed, and elaborated on by our primary sources who have direct knowledge of the compromised Supermicro hardware. Joe FitzPatrick was not one of these 17 individual primary sources that included company insiders and government officials, and his direct quote in the story describes a hypothetical example of how a hardware attack might play out, as the story makes clear."

Update: This article has been updated to clarify that the first Bloomberg article was published in Businessweek and the second article was published only on Bloomberg's website. Both articles were written by the same journalists. This article has also been updated to add the Bloomberg statement on Joe FitzPatrick, and telecom companies statements.